Pith. sign in

WAM4D: Fast 4D World Action Model via Spatial Register Tokens

1 Pith paper cite this work. Polarity classification is still indexing.

1 Pith paper citing it
abstract

World action models (WAMs) have recently shown promise in jointly modeling future observations and executable robot actions. However, most existing WAMs still operate in 2D video or latent spaces, where visually plausible rollouts miss the 3D spatial constraints and occluded contact geometry required for precise manipulation. While geometric foundation models offer strong priors for recovering dense 3D structure and motion from visual observations, forcing WAMs to predict the dense 4D representation introduces costly geometric decoding and slows down causal action generation. To address the trade-off, we present WAM4D, a fast 4D world action model that uses lightweight spatial register tokens as training-time future-depth readouts to transfer pretrained geometric priors into a causal video-action transformer, then removes the register branch for lightweight action inference. To prevent non-causal shortcuts, we further design causal mixture attention for the Mixture-of-Transformers (MoT) WAM backbone, defining modality-specific visibility among video, action, and geometry tokens. Comprehensive experiments on RoboTwin 2.0 and challenging real-world manipulation tasks show that WAM4D improves spatial consistency and achieves competitive action prediction while maintaining efficient inference.

fields

cs.CV 1

years

2026 1

verdicts

CONDITIONAL 1

representative citing papers

citing papers explorer

Showing 1 of 1 citing paper.

  • SUV: Future Scene Understanding as Video Generation for End-to-End Driving cs.CV · 2026-08-04 · conditional · none · ref 114 · internal anchor

    One shared video generator predicts future RGB, semantics, depth, and instance tracks simultaneously, and a separate action expert reads those latents to output the ego trajectory, reaching 91.0 EPDMS on NAVSIM-v2 navtest.