Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Robustness in world action models can be steered at inference time without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:37 UTC pith:O2T4BEDX

load-bearing objection Worth a real look: the separability-predicts-steerability claim is solid, but the LQR contribution is undersold and under-validated. the 3 major comments →

arxiv 2607.14943 v1 pith:O2T4BEDX submitted 2026-07-16 cs.RO cs.AIcs.LGcs.SYeess.SYmath.OC

Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control

Jihoon Hong , Julian Skifstad , Qiyue Dai , Alice Chan , Glen Chou This is my paper
classification cs.RO cs.AIcs.LGcs.SYeess.SYmath.OC
keywords world action modelsmechanistic interpretabilityactivation steeringlinear quadratic regulatorrobustnesscontrastive directionsdiffusion transformerlinear separability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether the internal activations of World Action Models (WAMs) — video-generative robot policies — carry simple, detectable structure for robustness-relevant perturbations such as camera shifts, gripper changes, and image noise. It shows that in some architectures, activations from successful versus unsuccessful rollouts are linearly separable in a low-dimensional subspace, and that this separability predicts whether activation steering will help. It then introduces WA-LQR, a reduced-order LQR controller that treats the diffusion-transformer block activations as a locally linear dynamical system, tracks a contrastive feature setpoint, and corrects deviations with closed-loop feedback. On Cosmos-Policy and DiT4DiT, WA-LQR transfers contrastive directions across tasks and improves success rates by up to 41% over unsteered and prompt-steering baselines, while the non-separable LingBot-VA model shows no such gains. The paper argues that hinge loss on three principal components is a practical diagnostic for steerability.

Core claim

Mechanistically comparing activations from clean versus perturbed or successful versus unsuccessful rollouts, the paper finds that robustness-relevant factors are sometimes encoded as low-dimensional, linearly separable directions in WAM activation space. This structure is architecture- and perturbation-dependent: Cosmos-Policy and DiT4DiT exhibit strong linear separability, while LingBot-VA does not. The central discovery is that this separability is not merely descriptive — it predicts steering outcomes and can be exploited directly. Contrastive activations yield steering vectors, and local linearity of the diffusion-transformer block dynamics within a low-dimensional contrastive subspace

What carries the argument

The central objects are contrastive activation directions (differences between hidden states from paired nominal/perturbed inputs), a low-dimensional latent subspace built by randomized SVD of those directions, and an SVM hinge loss on the top three principal components that quantifies linear separability. WA-LQR (World-Action Linear Quadratic Regulator) treats the steered diffusion-transformer block as the locally linear system δz_{l+1,t,τ} ≈ Ã_{l,t} δz_{l,t,τ} + B̃_{l,t} δu_{l,t,τ}, with Jacobian-vector products used to avoid materializing full Jacobians. For each action-chunk timestep and denoising timestep it solves an L-step LQR tracking problem that penalizes deviations from a contrast

Load-bearing premise

The load-bearing premise is that robustness-relevant perturbations live in a low-dimensional contrastive subspace and that the projected diffusion-transformer block dynamics stay approximately linear over the steering corrections actually applied, allowing one nominal Jacobian pair to be reused across inputs and tasks.

What would settle it

Measure the norm of the actual WA-LQR corrections injected during a rollout, feed perturbations of that size through the diffusion-transformer block, and compare the actual latent transition with the linear approximation used to compute the gains; if cosine similarity or magnitude ratio departs substantially from 1 in that regime, the linearity premise is violated and WA-LQR's gain computation no longer tracks the feature setpoint.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A cheap diagnostic: measuring hinge loss on the top three principal components of contrastive activation directions predicts whether inference-time steering will help a given model and perturbation type.
  • Cross-task transfer: contrastive directions and LQR gains computed on one LIBERO-10 task can be applied to other tasks in the same feature cluster, improving success rates without finetuning.
  • Closed-loop beats open-loop when oversteering is a risk: WA-LQR modulates correction magnitude based on realized activations, whereas activation addition needs careful tuning of the steering strength γ and can degrade performance.
  • Architecture matters: models like LingBot-VA, whose activations are not linearly separable under perturbations, do not benefit from these training-free steering methods, so robustness must be designed in rather than steered in.
  • On the evaluated WAMs, prompt steering is ineffective (matching or degrading baseline), while activation steering adds value.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to scan a candidate WAM's activations on a small perturbed set before deployment; low hinge loss would suggest steering is viable, but the paper leaves open how to discover shared feature clusters automatically without per-setting mechanistic analysis.
  • The same reduced-order latent dynamics could support other linear-control tools beyond LQR, such as disturbance observers or model-predictive control, which the paper does not explore.
  • Because linearity was validated with small random perturbations, the method's behavior at the actual steering magnitudes used at inference is a testable gap; if those corrections leave the linear regime, the reported robustness gains could weaken sharply.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies activation-space geometry in three World Action Models (Cosmos-Policy, DiT4DiT, LingBot-VA) under camera-orientation, gripper-position, and Gaussian-noise perturbations on LIBERO-10. It finds low-dimensional linear separability for Cosmos-Policy and DiT4DiT but not LingBot-VA, proposes a hinge-loss-based steerability predictor, uses contrastive directions for open-loop ActAdd, and introduces WA-LQR, a reduced-order LQR that projects activations into a rank-64 SVD subspace and steers toward a feature setpoint using locally linearized DiT block dynamics. Experiments report robustness improvements on Cosmos-Policy for camera and gripper shifts, mixed results on Gaussian noise, and no meaningful gains on LingBot-VA, consistent with the mechanistic predictions.

Significance. If the claims hold, the paper contributes a useful diagnostic—separability loss as a steerability predictor—and a training-free feedback steering method for WAMs, which is a relatively new area. The architecture-dependence finding is valuable, and the evaluation spans three models and three perturbation classes. The negative LingBot-VA result is an honest strength. However, the empirical support for the optimal-control component is incomplete: the linearity assumption is validated only with random perturbations, task groups are selected post hoc, and statistical significance is not reported. The correlation in Fig. 7 is moderate (r=0.63), so calling it a 'strong predictor' overstates the evidence.

major comments (3)
  1. [Sec. 6.1, Eq. (11), Fig. 5(a)] The reduced-order linearity assumption underlying WA-LQR is validated only with random perturbations ε whose norm scales with ||x_l,t||, not with the actual contrastive direction v^z_{l,t} at the magnitudes α_{l,t,τ} produced by Eq. (14). Fig. 5(a) is shown only for camera orientation, and a rank-64 projection can discard activation drift orthogonal to the contrastive subspace. The reuse of a single Jacobian pair (Ã_l,t, B̃_l,t) across tasks and inputs is also supported only by subspace overlap of top-16 singular vectors (Fig. 5(b)), not by prediction error on actual steered rollouts. Please add linearity checks along the steering direction at realized LQR strengths for all three perturbation classes, and provide an ablation showing that the closed-loop feedback term, rather than the open-loop setpoint term, is responsible for the reported gains.
  2. [Sec. 6.2, Tables 1–3] Task groups are selected after inspecting pairwise separability (Sec. 4, Figs. 18–20), and only tasks within these groups are evaluated. The abstract claims WA-LQR 'generalizes contrastive directions to new tasks,' but the reported averages are over a chosen subset; e.g., Table 1 camera orientation average excludes tasks 1,3,6,7,8, and Gaussian noise average excludes tasks 2,3,5,8,9. This post-hoc selection weakens the transfer claim. Please report results on all tasks (or at least on all non-source tasks) or specify a pre-defined grouping rule; otherwise the generalization statement is not supported.
  3. [Abstract, Sec. 4, Fig. 7] The claim that 'linear separability loss is a strong predictor of steering performance' is supported by r=0.63, which is a moderate correlation, and it is computed on the same tasks and contrastive definitions used to construct the steering directions. Several per-task comparisons in Tables 1–3 are within one standard error (e.g., Table 1 camera Task0→Task4: WA-LQR 13.3±6.2 vs. no steering 10.0±5.5; gripper Task1→Task3: 60.0±8.9 vs. 46.7±9.1). Please provide confidence intervals for the correlation, per-perturbation correlation values, and significance tests for the main success-rate differences. This is needed to support the 'strong predictor' and 'improves robustness' claims.
minor comments (6)
  1. [Table 3] The row 'T0→T255.0%' appears to be a formatting error; it should read 'T0→T2 55.0%'.
  2. [Table 2] The entry 'T0→T1 69.6%' lacks the error bar format used in all other cells; please make the table formatting consistent.
  3. [Fig. 5(a)] The caption says 'under change in camera orientation,' but the text describes random perturbation ε. Please clarify whether the linearity check is performed on clean or camera-perturbed activations.
  4. [Sec. 4 and Sec. 5.1] Section 4 says activations are average-pooled over action-chunk timesteps, while Sec. 5.1 defines per-chunk contrastive directions. Please reconcile this notation.
  5. [App. D.2] LQR hyperparameters (λ, R_init, R_final, τ_R, SVD rank) are selected by a per-source-task search. An sensitivity analysis analogous to App. C (for ActAdd γ) would strengthen the claim that WA-LQR is robust to these choices.
  6. [Header / Reproducibility] The header lists website/GitHub/video icons, but no URLs appear in the text. Please include links to the code and project page for reproducibility.

Circularity Check

0 steps flagged

No significant circularity: the derivation is self-contained and the central claims are validated by external benchmarks and held-out task transfer, not entailed by the definitions.

full rationale

The paper's main derivations are not circular by construction. The contrastive directions (Eqs. 8-10, 12) are computed from the same positive/negative activation sets used to define the target feature, so pushing activations along those directions is partly a self-consistency check in isolation. However, the load-bearing claims are (i) that low hinge loss predicts steerability and (ii) that WA-LQR improves robustness. Both are evaluated on LIBERO-10 success rates, an external metric, including cross-task transfer (Table 1, e.g., Task 0 to Task 2), and the hinge-loss/steerability correlation (Fig. 7, r=0.63) is an empirical finding, not an identity. The reduced-order dynamics assumption in Eq. 11 is validated in Sec. 6.1 with random perturbations; the concern that this does not test actual steering magnitudes is a support gap, not a definitional reduction. Citations [51] and [55] are from the same author group, but the local-linearity premise is independently tested (Fig. 5a, 5b) and WA-LQR is tested against external baselines, so the self-citations are not load-bearing. The feature setpoint beta* = lambda ||e_z|| involves a tuned lambda, but tuning a scalar gain does not make the success-rate outcome a mathematical consequence of the input equations. No step in the derivation reduces to its own inputs; hence score 0.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The paper introduces no new physical entities. Its free parameters are tuning knobs (lambda, gamma, R-schedule, rank, C) that affect the steering results materially; its axioms are the standard LQR machinery plus domain assumptions about low-dimensional linear structure and transferability in WAM activations. The counts here are the most honest measure of what the paper adds versus what it takes from prior literature.

free parameters (5)
  • lambda (feature setpoint strength) = not reported
    In Eq. (12), beta* = lambda ||e_z|| sets the target feature strength. Tuned via hyperparameter search (App. D.2); analogous to gamma in ActAdd, which shows a sharp peak at gamma=0.1 in App. C.
  • LQR cost schedule R_init, R_final, tau_R = not reported
    r(tau) = min(R_final, R_init exp(tau/tau_R)) in Sec. 5.2 controls steering strength and decay across action chunks. Values are not given; they are selected by hyperparameter search and materially affect success rates.
  • gamma (ActAdd steering strength) = 0.1 for Cosmos Gaussian noise (App. C)
    Open-loop activation addition strength in Eq. (10). Table 5 shows extreme sensitivity: performance collapses to 0% outside a narrow band, so the reported ActAdd numbers depend strongly on this tuned value.
  • SVD subspace rank dz = 64 for all models
    Rank of the projection matrix V_l,t in Sec. 5.2. Chosen by hand; affects how much contrastive information is retained and the dimension of the subsequent LQR problem.
  • SVM regularization C = 10 (App. D.1)
    Used in the hinge-loss separability metric (Eq. 15). Reasonable but arbitrary; affects the predictor values reported in Fig. 7 and the appendices.
axioms (6)
  • standard math LQR/Riccati theory gives the optimal linear feedback for the quadratic cost in Eq. (1)-(2).
    Invoked in Sec. 3 and used to compute gains in Sec. 5.2. Standard textbook result.
  • domain assumption Contrastive directions from successful vs unsuccessful (or clean vs noisy) rollouts encode a robustness feature that, when amplified, improves success (Eq. 8, App. A).
    The whole steering objective rests on the semantic interpretation of D+ as 'desired' and D- as 'undesired'. This is not derived; it is assumed and validated only through downstream success rates.
  • domain assumption Top-64 SVD subspace of contrastive directions preserves all steering-relevant activation information (Sec. 5.2, Fig. 6).
    Fig. 6 shows high explained variance for camera orientation on Cosmos, but the paper does not prove that the truncated directions are the only action-relevant ones, and the rank is fixed at 64 across all models and perturbations.
  • domain assumption Projected DiT block dynamics are locally linear: delta z_{l+1} ≈ eA_l,t delta z_l + eB_l,t delta u_l (Eq. 11).
    Validated in Fig. 5 for small random perturbations, but actual steering interventions may be larger; no stated bound on the perturbation norm for which the linearization remains accurate.
  • domain assumption Jacobians and subspaces computed from one task's nominal rollouts transfer to other tasks (Fig. 5b).
    Fig. 5b shows overlap of top-16 right singular vectors over 25 random inputs from 5 tasks, but the transfer is not guaranteed and the paper limits its evaluations to empirically selected task groups.
  • domain assumption Hinge loss in the top-3 PCA subspace is a faithful proxy for full-space linear separability of the feature (Eq. 7).
    The paper deliberately uses 3 PCs because high-dimensional data are trivially separable; this makes the metric a deliberate modeling choice. It correlates with steering outcomes (r=0.63) but is not a formal guarantee.

pith-pipeline@v1.3.0-alltime-deepseek · 33106 in / 13770 out tokens · 143374 ms · 2026-08-02T00:37:05.829081+00:00 · methodology

0 comments
read the original abstract

World Action Models (WAMs) enable semantically- and physically-informed control but are brittle under distribution shift. In this work, we use mechanistic interpretability to study how robustness-relevant perturbations are represented in WAM activation space. Comparing activations across successful and unsuccessful rollouts, we find some WAM architectures exhibit low-dimensional linear separability for robustness-critical features, while others do not. This motivates the use of contrastive activation directions for training-free WAM steering. We also show that local linearity in WAM activation dynamics enables efficient feedback steering via model-based optimal control, yielding World-Action Linear Quadratic Regulator (WA-LQR), a minimally-invasive reduced-order LQR controller. Via mechanistic evaluations, we predict strong steerability in the Cosmos-Policy and DiT4DiT models but weak steerability in LingBot-VA, consistent with steering intervention results. On Cosmos-Policy and DiT4DiT, WA-LQR generalizes contrastive directions to new tasks and improves robustness to camera, gripper, and visual-noise perturbations over unsteered and prompt steering baselines.

Figures

Figures reproduced from arXiv: 2607.14943 by Alice Chan, Glen Chou, Jihoon Hong, Julian Skifstad, Qiyue Dai.

Figure 1
Figure 1. Figure 1: WA-LQR makes World Action Models more robust to perturbations including gripper [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: On Task 0 of LIBERO-10 [22], we evaluate (a) activations corresponding to noise￾perturbed and clean inputs on the first, best intermediate, and final DiT block residual stream for Cosmos-Policy 2B and DiT4DiT [2, 6]. Activations are projected onto the top three principal com￾ponents, with the reported SVM (gray) and hinge loss. (b) Repeated for camera perturbations. where h is the embedding of the task pro… view at source ↗
Figure 3
Figure 3. Figure 3: Pairwise separability for Cosmos-Policy under [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cumulative variance of latent contrastive vec [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) shows the cosine similarity and the magnitude ratio between Pl+1,tϕ (l)  xl,t + P T l,tϵ, t, h and Pl+1,tϕ (l) (xl,t, t, h) + A˜ l,tϵ for random perturbation ϵ whose norm is proportional to that of ∥xl,t∥. The cosine similarities and magnitude ratios remain close to 1 throughout inference, demon- (𝐚) (𝐛) 𝟏.𝟎 𝟎.𝟖 𝟎.𝟔 𝟎.𝟒 𝟎.𝟐 𝟎.𝟎 (𝐥, 𝐭)=(𝟔,𝟎) (𝐥, 𝐭)=(𝟏𝟒,𝟐) (𝐥, 𝐭)=(𝟐𝟓,𝟒) 𝐑𝐚𝐧𝐝𝐨𝐦 𝐂 𝐨 𝐬 𝐢 𝐧 𝐞 𝐒 𝐢 𝐦 𝐢 𝐥 𝐚 𝐫… view at source ↗
Figure 7
Figure 7. Figure 7: Hinge loss vs. steering performance across tasks and models, with reported line of best fit (red) and correlation coefficient. We leverage the mechanistic insights from Sec. 4 to inform steering objectives designed to improve robustness to OOD perturbations in Cosmos-Policy 2B [2], DiT4DiT [6], and LingBot-VA [7], using WA-LQR (Sec. 5.2) and a variant of simple activation addition adapted to WAMs (Sec. 5.1… view at source ↗
Figure 8
Figure 8. Figure 8: Snapshots from rollouts where steering enables task success despite unsteered failure, [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Cosmos-Policy noise corruption separation for all LIBERO-10 tasks [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Cosmos-Policy camera perturbation separation for all LIBERO-10 tasks. [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cosmos-Policy gripper perturbation separation for all LIBERO-10 tasks. [PITH_FULL_IMAGE:figures/full_fig_p024_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: LingBot-VA camera perturbation separation for all LIBERO-10 tasks (Action Module). [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: LingBot-VA gripper perturbation separation for all LIBERO-10 tasks (Action Module). [PITH_FULL_IMAGE:figures/full_fig_p025_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: LingBot-VA noise corruption separation for all LIBERO-10 tasks (Action Module). [PITH_FULL_IMAGE:figures/full_fig_p025_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: LingBot-VA camera orientation perturbation separation for LIBERO-10 task 0, 2, 4, 5, 9 [PITH_FULL_IMAGE:figures/full_fig_p026_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: LingBot-VA noise corruption separation for LIBERO-10 task 0, 1, 4, 6, 7 (Video Mod [PITH_FULL_IMAGE:figures/full_fig_p027_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: LingBot-VA gripper perturbation separation for LIBERO-10 task 1, 2, 3, 7, 9 (Video [PITH_FULL_IMAGE:figures/full_fig_p028_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Pairwise separability for Cosmos-Policy with Gaussian noise corruption. [PITH_FULL_IMAGE:figures/full_fig_p029_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Pairwise separability for Cosmos-Policy with gripper position perturbation. [PITH_FULL_IMAGE:figures/full_fig_p030_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Pairwise separability for Cosmos-Policy with camera position perturbation. [PITH_FULL_IMAGE:figures/full_fig_p031_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Cosmos-Policy Task 0 noise corruption 32 [PITH_FULL_IMAGE:figures/full_fig_p032_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Cosmos-Policy Task 0 noise corruption 33 [PITH_FULL_IMAGE:figures/full_fig_p033_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Cosmos-Policy Task 1 noise corruption 34 [PITH_FULL_IMAGE:figures/full_fig_p034_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Cosmos-Policy Task 1 noise corruption 35 [PITH_FULL_IMAGE:figures/full_fig_p035_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Cosmos-Policy Task 0 camera perturbation [PITH_FULL_IMAGE:figures/full_fig_p036_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Cosmos-Policy Task 0 camera perturbation [PITH_FULL_IMAGE:figures/full_fig_p037_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Cosmos-Policy Task 1 camera perturbation [PITH_FULL_IMAGE:figures/full_fig_p038_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Cosmos-Policy Task 1 camera perturbation [PITH_FULL_IMAGE:figures/full_fig_p039_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Feature separability: DiT4DiT Gaussian noise corruption for tasks 0-2. [PITH_FULL_IMAGE:figures/full_fig_p040_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Feature separability: DiT4DiT gripper perturbation for tasks 0-3. [PITH_FULL_IMAGE:figures/full_fig_p041_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: DiT4DiT Task 0 noise corruption 42 [PITH_FULL_IMAGE:figures/full_fig_p042_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: DiT4DiT Task 1 noise corruption PC1 PC2 PC3 Block 0 avg loss: 0.8109 Tasks 1&2 First Cond. / Task task 1 positive task 1 negative task 2 positive task 2 negative PC1 PC2 PC3 Block 5 avg loss: 0.0000 Best PC1 PC2 PC3 Block 15 avg loss: 0.0113 Last PC1 PC2 PC3 Block 0 avg loss: 0.8542 Tasks 1&0 PC1 PC2 PC3 Block 4 avg loss: 0.0000 PC1 PC2 PC3 Block 15 avg loss: 0.0000 PC1 PC2 PC3 Block 0 avg loss: 0.9041 Ta… view at source ↗
Figure 33
Figure 33. Figure 33: Pairwise Gaussian noise corruption feature separation on DiT4DiT. [PITH_FULL_IMAGE:figures/full_fig_p043_33.png] view at source ↗
Figure 34
Figure 34. Figure 34: Pairwise gripper perturbation feature separation on DiT4DiT. [PITH_FULL_IMAGE:figures/full_fig_p044_34.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FBFM: A Training-Free Asynchronous Feedback Mechanism for Flow-Matching in World-Action Models Execution

    cs.RO 2026-07 conditional novelty 6.0

    A training-free masked pseudoinverse-guidance mechanism injects newly observed states and committed actions into the active flow-matching chunk of world-action models, improving long-horizon execution on selected mani...

Reference graph

Works this paper leans on

71 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    M. J. Kim, Y . Gao, T.-Y . Lin, Y .-C. Lin, Y . Ge, G. Lam, P. Liang, S. Song, M.-Y . Liu, C. Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    Pith/arXiv arXiv 2026

  3. [3]

    S. Ye, Y . Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y . L. Tan, C. Zhu, J. Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  4. [4]

    A. Ye, B. Wang, C. Ni, G. Huang, G. Zhao, H. Li, H. Li, J. Li, J. Lv, J. Liu, et al. Gigaworld- policy: An efficient action-centered world–action model.arXiv preprint arXiv:2603.17240, 2026

    arXiv 2026

  5. [5]

    Zhang, Z

    Z. Zhang, Z. Li, B. Rahmati, R. H. Yang, Y . Ma, A. Rasouli, S. Pakdamansavoji, Y . Wu, L. Zhang, T. Cao, et al. Do world action models generalize better than vlas? a robustness study.arXiv preprint arXiv:2603.22078, 2026

    Pith/arXiv arXiv 2026

  6. [6]

    T. Ma, J. Zheng, Z. Wang, C. Jiang, A. Cui, J. Liang, and S. Yang. Dit4dit: Jointly modeling video dynamics and actions for generalizable robot control.arXiv preprint arXiv:2603.10448, 2026

    arXiv 2026

  7. [8]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  8. [9]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  9. [10]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  10. [11]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  11. [12]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  12. [13]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    Pith/arXiv arXiv 2025

  13. [14]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  14. [15]

    B. Hou, G. Li, J. Jia, T. An, X. Guo, S. Leng, H. Geng, Y . Ze, T. Harada, P. Torr, et al. World model for robot learning: A comprehensive survey.arXiv preprint arXiv:2605.00080, 2026. 12

    Pith/arXiv arXiv 2026

  15. [16]

    S. Wang, J. Shi, Z. Fu, X. He, F. Liu, C. Yang, Y . Zhou, Z. Fei, J. Gong, J. Fu, et al. World action models: The next frontier in embodied ai.arXiv preprint arXiv:2605.12090, 2026

    Pith/arXiv arXiv 2026

  16. [17]

    Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuurmans, and P. Abbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    2023

  17. [18]

    Y . Wen, J. Lin, Y . Zhu, J. Han, H. Xu, S. Zhao, and X. Liang. Vidman: Exploiting implicit dynamics from video diffusion model for effective robot manipulation.Advances in Neural Information Processing Systems, 37:41051–41075, 2024

    2024

  18. [19]

    Y . Hu, Y . Guo, P. Wang, X. Chen, Y .-J. Wang, J. Zhang, K. Sreenath, C. Lu, and J. Chen. Video prediction policy: A generalist robot policy with predictive visual representations.arXiv preprint arXiv:2412.14803, 2024

    Pith/arXiv arXiv 2024

  19. [20]

    Y . Jia, J. Liu, S. Liu, R. Zhou, W. Yu, Y . Yan, X. Chi, Y . Guo, B. Shi, and S. Zhang. Video2act: A dual-system video diffusion policy with robotic spatio-motional modeling.arXiv preprint arXiv:2512.03044, 2025

    arXiv 2025

  20. [21]

    L. Li, Q. Zhang, Y . Luo, S. Yang, R. Wang, F. Han, M. Yu, Z. Gao, N. Xue, X. Zhu, et al. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Pith/arXiv arXiv 2026

  21. [22]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023

    2023

  22. [23]

    X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kir- mani, et al. Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024

    Pith/arXiv arXiv 2024

  23. [24]

    Z. Wang, Z. Zhou, J. Song, Y . Huang, Z. Shu, and L. Ma. Vlatest: Testing and evaluating vision-language-action models for robotic manipulation.Proceedings of the ACM on Software Engineering, 2(FSE):1615–1638, 2025

    2025

  24. [25]

    Pumacay, I

    W. Pumacay, I. Singh, J. Duan, R. Krishna, J. Thomason, and D. Fox. The colosseum: A bench- mark for evaluating generalization for robotic manipulation.arXiv preprint arXiv:2402.08191, 2024

    Pith/arXiv arXiv 2024

  25. [26]

    S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025

    Pith/arXiv arXiv 2025

  26. [27]

    J. Zhou, K. Ye, J. Liu, T. Ma, Z. Wang, R. Qiu, K.-Y . Lin, Z. Zhao, and J. Liang. Exploring the limits of vision-language-action manipulations in cross-task generalization, 2025. URL https://arxiv.org/abs/2505.15660

    arXiv 2025

  27. [28]

    J. Liu, F. Gao, B. Wei, X. Chen, Q. Liao, Y . Wu, C. Yu, and Y . Wang. What can rl bring to vla generalization? an empirical study.Advances in Neural Information Processing Systems, 38: 97121–97151, 2026

    2026

  28. [29]

    S. Tan, K. Dou, Y . Zhao, and P. Kr ¨ahenb¨uhl. Interactive post-training for vision-language- action models.arXiv preprint arXiv:2505.17016, 2025

    Pith/arXiv arXiv 2025

  29. [30]

    Y . Guo, J. Zhang, X. Chen, X. Ji, Y .-J. Wang, Y . Hu, and J. Chen. Improving vision-language- action model with online reinforcement learning. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 15665–15672. IEEE, 2025

    2025

  30. [31]

    T. Yuan, Z. Dong, Y . Liu, and H. Zhao. Fast-wam: Do world action models need test-time future imagination?arXiv preprint arXiv:2603.16666, 2026. 13

    Pith/arXiv arXiv 2026

  31. [32]

    Bereska and S

    L. Bereska and S. Gavves. Mechanistic interpretability for AI safety - a review.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URLhttps://openreview.net/ forum?id=ePUVetPKu6. Survey Certification, Expert Certification

    2024

  32. [33]

    Sharkey, B

    L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimersheim, A. Ortega, J. Bloom, et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025

    Pith/arXiv arXiv 2025

  33. [34]

    Elhage, T

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wat- tenberg, and C. Olah. Toy models of superposition. (arXiv:2209.10652), 2022. doi:10.48550/ arXiv.2209.10652. URLhttp://arxiv.org/abs/2209.10652. arXiv:2209.10652

    Pith/arXiv arXiv doi:10.48550/arxiv.2209.10652 2022

  34. [35]

    K. Park, Y . J. Choe, and V . Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658, 2023

    Pith/arXiv arXiv 2023

  35. [36]

    Marks and M

    S. Marks and M. Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. InFirst Conference on Language Modeling, 2024. URLhttps://openreview.net/forum?id=aajyHYjjsk

    2024

  36. [37]

    A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.- K. Dombrowski, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023

    Pith/arXiv arXiv 2023

  37. [38]

    B. W. Lee, I. Padhi, K. N. Ramamurthy, E. Miehling, P. Dognin, M. Nagireddy, and A. Dhu- randhar. Programming refusal with conditional activation steering. InThe Thirteenth Inter- national Conference on Learning Representations, 2025. URLhttps://openreview.net/ forum?id=Oi47wc10sm

    2025

  38. [39]

    Dathathri, A

    S. Dathathri, A. Madotto, J. Lan, J. Hung, E. Frank, P. Molino, J. Yosinski, and R. Liu. Plug and play language models: A simple approach to controlled text generation. InInternational Conference on Learning Representations, 2020. URLhttps://openreview.net/forum? id=H1edEyBKDS

    2020

  39. [40]

    K. Li, O. Patel, F. Vi´egas, H. Pfister, and M. Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36:41451–41530, 2023

    2023

  40. [41]

    A. M. Turner, L. Thiergart, G. Leech, D. Udell, U. Mini, and M. MacDiarmid. Activation addition: Steering language models without optimization. 2024

    2024

  41. [42]

    Rimsky, N

    N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner. Steering llama 2 via contrastive activation addition. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 15504–15522, 2024

    2024

  42. [43]

    Arditi, O

    A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda. Refusal in language models is mediated by a single direction.Advances in Neural Information Processing Systems, 37:136037–136083, 2024

    2024

  43. [44]

    Rodriguez, A

    P. Rodriguez, A. Blaas, M. Klein, L. Zappella, N. Apostoloff, X. Suau, et al. Controlling language and diffusion models by transporting activations. InInternational Conference on Learning Representations, volume 2025, pages 89812–89855, 2025

    2025

  44. [45]

    Z. Wu, A. Arora, Z. Wang, A. Geiger, D. Jurafsky, C. D. Manning, and C. Potts. Reft: Repre- sentation finetuning for language models.Advances in Neural Information Processing Systems, 37:63908–63962, 2024

    2024

  45. [46]

    H. M. Vu and T. M. Nguyen. Angular steering: Behavior control via rotation in activa- tion space. (arXiv:2510.26243), Oct. 2025. doi:10.48550/arXiv.2510.26243. URLhttp: //arxiv.org/abs/2510.26243. arXiv:2510.26243. 14

    doi:10.48550/arxiv.2510.26243 2025

  46. [47]

    Bhargava, C

    A. Bhargava, C. Witkowski, S.-Z. Looi, and M. Thomson. What’s the magic word? a control theory of llm prompting.arXiv preprint arXiv:2310.04444, 2023

    Pith/arXiv arXiv 2023

  47. [48]

    L. Kong, H. Wang, W. Mu, Y . Du, Y . Zhuang, Y . Zhou, Y . Song, R. Zhang, K. Wang, and C. Zhang. Aligning large language models with representation editing: A control perspective. (arXiv:2406.05954), Nov. 2024. doi:10.48550/arXiv.2406.05954. URLhttp://arxiv.org/ abs/2406.05954. arXiv:2406.05954

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.05954 2024

  48. [49]

    Cheng and C

    E. Cheng and C. A. Alonso. Linearly controlled language generation with performative guarantees. (arXiv:2405.15454), Sept. 2025. doi:10.48550/arXiv.2405.15454. URLhttp: //arxiv.org/abs/2405.15454. arXiv:2405.15454

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.15454 2025

  49. [50]

    D. V . Nguyen, H. M. Vu, N. Y . Pham, L. Zhang, and T. M. Nguyen. Activation steering with a feedback controller. (arXiv:2510.04309), Oct. 2025. doi:10.48550/arXiv.2510.04309. URL http://arxiv.org/abs/2510.04309. arXiv:2510.04309

    Pith/arXiv arXiv doi:10.48550/arxiv.2510.04309 2025

  50. [51]

    Skifstad, X

    J. Skifstad, X. A. Yang, and G. Chou. Local linearity of llms enables activation steering via model-based linear optimal control.arXiv preprint arXiv:2604.19018, 2026

    Pith/arXiv arXiv 2026

  51. [52]

    Rodriguez, M

    P. Rodriguez, M. Klein, E. Gualdoni, V . Maiorca, A. Blaas, L. Zappella, M. Cuturi, and X. Suau. Lineas: End-to-end learning of activation steering with a distributional loss.arXiv preprint arXiv:2503.10679, 2025

    arXiv 2025

  52. [53]

    Facchiano, S

    S. Facchiano, S. Saravalle, M. Migliarini, E. De Matteis, A. Sampieri, A. Pilzer, E. Rodol `a, I. Spinelli, L. Franco, and F. Galasso. Video unlearning via low-rank refusal vector.arXiv preprint arXiv:2506.07891, 2025

    arXiv 2025

  53. [54]

    Ekin and Y

    Y . Ekin and Y . Gandelsman. The unreasonable effectiveness of text embedding interpolation for continuous image steering.arXiv preprint arXiv:2603.17998, 2026

    arXiv 2026

  54. [55]

    J. Hong, A. Chan, Q. Dai, J. Skifstad, and G. Chou. Activation steering of video generation models via reduced-order linear optimal control. 2026

    2026

  55. [56]

    H ¨aon, K

    B. H ¨aon, K. C. Stocking, I. Chuang, and C. Tomlin. Mechanistic interpretability for steering vision-language-action models. In J. Lim, S. Song, and H.-W. Park, editors,Proceedings of The 9th Conference on Robot Learning, volume 305 ofProceedings of Machine Learning Research, pages 2743–2762. PMLR, 27–30 Sep 2025. URLhttps://proceedings.mlr. press/v305/h...

    2025

  56. [57]

    Buurmeijer, C

    H. Buurmeijer, C. A. Alonso, A. Swann, and M. Pavone. Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026

    arXiv 2026

  57. [58]

    Mitra, Y

    C. Mitra, Y . Luo, R. Saravanan, D. Niu, A. Pai, J. Thomason, T. Darrell, A. Anwar, D. Ra- manan, and R. Herzig. Mechanistic finetuning of vision-language-action models via few-shot demonstrations.arXiv preprint arXiv:2511.22697, 2025

    arXiv 2025

  58. [59]

    Swann, L

    A. Swann, L. McGranahan, H. Buurmeijer, M. Kennedy III, and M. Schwager. Sparse autoencoders reveal interpretable and steerable features in vla models.arXiv preprint arXiv:2603.19183, 2026

    Pith/arXiv arXiv 2026

  59. [60]

    Zhang, M

    H. Zhang, M. Xu, A. Dhafer, S. Yue, H. Dong, and Z. D. Hao. Embodied interpretability: Link- ing causal understanding to generalization in vision-language-action models.arXiv preprint arXiv:2605.00321, 2026

    Pith/arXiv arXiv 2026

  60. [61]

    Grant, X

    B. Grant, X. Zhao, and P. Wang. Not all features are created equal: A mechanistic study of vision-language-action models.arXiv preprint arXiv:2603.19233, 2026

    arXiv 2026

  61. [62]

    M. A. Khan, N. Boskov, F. M. Anwar, and M. A. Khan. Controlling vision–language–action policies through sparse latent directions. InMechanistic Interpretability Workshop at NeurIPS 2025. 15

    2025

  62. [63]

    M. M. Miao, S. Kim, B. Yang, and L. Ungar. Contrastive conceptor activation steering (coast): Unlocking vision-language-action models through hidden states.arXiv preprint arXiv:2605.17144, 2026

    Pith/arXiv arXiv 2026

  63. [64]

    J. P. Hespanha.Linear systems theory. Princeton university press, 2018

    2018

  64. [65]

    Peebles and S

    W. Peebles and S. Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  65. [66]

    Marks and M

    S. Marks and M. Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets.arXiv preprint arXiv:2310.06824, Aug. 2024. doi:10.48550/arXiv.2310.06824. URLhttp://arxiv.org/abs/2310.06824

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.06824 2024

  66. [67]

    Huang, M

    C. Huang, M. M. Zhang, R. Azarcon, G. Chou, and Z. Kira. Maps: Preserving vision-language representations via module-wise proximity scheduling for better vision-language-action gener- alization.arXiv preprint arXiv:2511.19878, Nov. 2025. doi:10.48550/arXiv.2511.19878. URL http://arxiv.org/abs/2511.19878

    doi:10.48550/arxiv.2511.19878 2025

  67. [68]

    Kecman.Support V ector Machines – An Introduction, page 1–47

    V . Kecman.Support V ector Machines – An Introduction, page 1–47. Springer, Berlin, Heidel- berg, 2005. ISBN 9783540323846. doi:10.1007/10984697 1. URLhttps://doi.org/10. 1007/10984697_1

    work page doi:10.1007/10984697 2005

  68. [69]

    Halko, P.-G

    N. Halko, P.-G. Martinsson, and J. A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.SIAM review, 53(2):217–288, 2011

    2011

  69. [70]

    J. B. Rawlings, D. Q. Mayne, M. Diehl, et al.Model predictive control: theory, computation, and design, volume 2. Nob Hill Publishing Madison, WI, 2020

    2020

  70. [71]

    Fang and G

    J. Fang and G. Chou. Safe large-scale robust nonlinear mpc in milliseconds via reachability- constrained system level synthesis on the gpu.arXiv preprint arXiv:2604.07644, 2026

    Pith/arXiv arXiv 2026

  71. [72]

    steering away

    Z. Wang, Y . Chen, Y . Liu, J. Ye, P. Chen, C. Lu, S. Liu, B. Yu, and J. Jia. Vp- vla: Visual prompting as an interface for vision-language-action models.arXiv preprint arXiv:2603.22003, May 2026. doi:10.48550/arXiv.2603.22003. URLhttp://arxiv.org/ abs/2603.22003. arXiv:2603.22003. 16 Appendices In the following, we provide an overview of our appendices. ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2603.22003 2026