Pith. sign in

REVIEW 10 cited by

VaViM and VaVAM: Autonomous Driving through Video Generative Modeling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.15672 v1 pith:7DS6TEBO submitted 2025-02-21 cs.CV cs.AIcs.RO

Florent Bartoccioni , Elias Ramzi , Victor Besnier , Shashanka Venkataramanan , Tuan-Hung Vu , Yihong Xu , Loick Chambon , Spyros Gidaris
show 9 more authors
Serkan Odabas David Hurych Renaud Marlet Alexandre Boulch Mickael Chen Éloi Zablocki Andrei Bursuc Eduardo Valle Matthieu Cord
This is my paper
classification cs.CVcs.AIcs.RO
keywords drivingmodelvideovavimautonomousmodelsvavamauto-regressive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We explore the potential of large-scale generative video models for autonomous driving, introducing an open-source auto-regressive video model (VaViM) and its companion video-action model (VaVAM) to investigate how video pre-training transfers to real-world driving. VaViM is a simple auto-regressive video model that predicts frames using spatio-temporal token sequences. We show that it captures the semantics and dynamics of driving scenes. VaVAM, the video-action model, leverages the learned representations of VaViM to generate driving trajectories through imitation learning. Together, the models form a complete perception-to-action pipeline. We evaluate our models in open- and closed-loop driving scenarios, revealing that video-based pre-training holds promise for autonomous driving. Key insights include the semantic richness of the learned representations, the benefits of scaling for video synthesis, and the complex relationship between model size, data, and safety metrics in closed-loop evaluations. We release code and model weights at https://github.com/valeoai/VideoActionModel

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mirror Learning

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Fine-tuning a video diffusion model to perform cross-view perspective transfer, then labeling the generated first-person videos with an inverse dynamics model, yields behavior-cloning data that improves driving policies.

  2. Multiplayer Interactive World Models with Representation Autoencoders

    cs.CV 2026-07 accept novelty 7.0 of 10

    A 5B-parameter latent diffusion model generates real-time four-player Rocket League matches conditioned on all players' actions, staying stable far beyond its training horizon.

  3. Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features

    cs.RO 2026-08 conditional novelty 6.0 of 10

    An early-exit, quality-guided planner on a Wan2.2-5B video-diffusion backbone reaches 90.8 PDMS on NAVSIM in about 170 ms, without generating future video at deployment.

  4. Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

    cs.GR 2026-07 conditional novelty 6.0 of 10

    A feed-forward model reconstructs a layered, simulation-ready 3D Gaussian world from multi-view driving video in ~1.5 s, with quality approaching per-scene optimized reconstruction.

  5. WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A five-aspect, 24-metric benchmark, a 26K human-annotated dataset, and an AI evaluator show that today's driving world models cannot simultaneously look real, respect geometry, and behave safely.

  6. 3D and 4D World Modeling: A Survey

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A survey that defines 3D/4D world modeling, organizes methods into VideoGen, OccGen, and LiDARGen categories, and compiles datasets, metrics, and benchmark numbers.

  7. Back to the Features: DINO as a Foundation for Video World Models

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A world model trained in frozen DINOv2 latent space on 66M videos beats much larger pixel-space models on forecasting and physics benchmarks, and fine-tunes for planning.

  8. World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model

    cs.CV 2025-07 conditional novelty 5.0 of 10

    World4Drive couples multiple driving intentions with a latent world model to generate, score, and select trajectories, reporting state-of-the-art perception-free planning on nuScenes and NavSim.

  9. Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving

    cs.CV 2026-01 reject novelty 4.0 of 10

    A video-pretrained encoder plus simulator-distilled multimodal trajectory proposals scores 93.3 PDMS on NAVSIM v1 and 87.8 EPDMS on v2, but the v1 number is not the highest in the paper's own table.

  10. A Survey on Vision-Language-Action Models for Autonomous Driving

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A survey organizes vision-language-action models for autonomous driving into four stages, compares over 20 systems, and catalogs datasets, benchmarks, and open challenges.

Pith tools