The reviewed record of science sign in
Pith

arxiv: 2607.05352 · v1 · pith:456GODI3 · submitted 2026-07-06 · cs.CV · cs.AI· cs.LG

Multiplayer Interactive World Models with Representation Autoencoders

Anthony Hu , V\'aclav Volhejn , Adrien Ramanana Rahary , Chris Mulder , Aditya Makkar , Am\'elie Royer , Manu Orsini , Alyx Liao
show 19 more authors
Adam Jelley Eloi Alonso Florian Laurent Fredrik Nor\'en James Swingos Jan H\"unermann Kent Rollins Lucas Hosseini Matthieu Le Cauchois Maxim Peter Pim de Witte Tim Brown Vincent Micheli Moritz B\"ohle Gabriel de Marmiesse Viktoriia Sharmanska Lucia Specia Michael Black Patrick P\'erez
This is my paper

Reviewed by Pith2026-07-07 15:24 UTCglm-5.2pith:456GODI3open to challenge →

classification cs.CV cs.AIcs.LG
keywords world modelmultiplayerlatent diffusionrepresentation autoencoderdiffusion forcingaction conditioningRocket Leaguereal-time generation
0
0 comments X

The pith

Four-player world model runs real-time, stays stable for hours

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims the first interactive world model that conditions on the simultaneous action streams of four agents in a fast, physically complex environment. The system, called MIRA, is a 5-billion-parameter latent diffusion model trained on 10,000 hours of Rocket League gameplay. It renders all four players' perspectives at 20 frames per second on a single GPU and, despite being trained only on 4-second clips, produces rollouts that remain visually coherent and dynamically faithful for hours. The central mechanism is a combination of three design choices: a latent prediction space built on a frozen pretrained feature extractor (DINOv3), which prevents long-horizon drift; diffusion forcing, which trains the model on partially corrupted context so it can survive its own imperfect predictions at inference; and tiled multiplayer conditioning, which lets a single transformer jointly render all four views and attribute each player's actions to the correct car. The paper argues that these choices together yield a model that captures the game's underlying dynamics rather than merely memorizing visual patterns, supported by evidence that the model stays coherent under human control despite being trained only on bot play.

Core claim

The paper's central result is that conditioning a latent diffusion world model on four simultaneous action streams, predicting in the latent space of a frozen pretrained feature extractor, and training with diffusion forcing yields real-time, multi-agent rollouts that stay stable far beyond the training horizon. The pretrained feature extractor is the load-bearing ingredient for long-horizon stability: codecs built on from-scratch extractors reconstruct video more sharply but drift substantially over sustained rollouts, while the pretrained-extractor latent stays flat. Diffusion forcing is the second pillar, keeping teacher-forced rollouts from collapsing past the 4-second training window. T

What carries the argument

A representation autoencoder that compresses video into a compact latent by combining a frozen DINOv3 feature extractor with a learned linear bottleneck (2x spatial, 2x temporal downsampling), paired with a flow-matching diffusion transformer that predicts future latents autoregressively. Each frame receives an independent noise level during training (diffusion forcing), and the four players' views are tiled into a single grid so spatial attention can keep them mutually consistent.

If this is right

  • If the design transfers, agents could be trained or evaluated inside learned simulators for multi-agent tasks without running the real environment, reducing the cost of reinforcement learning.
  • The finding that pretrained feature extractors prevent rollout drift suggests that the smoothness of the prediction space, not just reconstruction quality, is what makes a latent suitable for autoregressive generation.
  • The action recoverability ratio (ARR) provides a reusable protocol for measuring whether generative models actually obey control inputs, addressing a gap between visual fidelity and dynamical correctness.
  • The two-stage training recipe (pretrain single-player, then warm-start multiplayer) offers a practical path for scaling multi-agent conditioning without requiring the full multiplayer compute budget from the start.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the model's stability under human control is genuine (not just interpolation within the bot's behavioral distribution), it would imply the model has learned a generalizable dynamics model rather than a policy imitation, which would be a stronger claim than the paper's single-bot training data would guarantee.
  • The emergent 'theory of mind' for unconditioned players could be tested more rigorously by training on data from multiple distinct bot policies and checking whether the model can interpolate between their playing styles at inference.
  • The drift resistance of pretrained feature extractors may connect to the spectral smoothness of self-supervised representations: nearby states mapping to nearby latents would absorb prediction errors rather than amplifying them, a hypothesis that could be tested by measuring the Lipschitz constant of the latent mapping.
  • The fact that the model generalizes beyond its training action distribution (staying stable when all cars sit still, or under human play) partially addresses the single-policy concern, but a systematic out-of-distribution action evaluation would strengthen the dynamics-learning claim.

Load-bearing premise

The model is trained entirely on data from a single bot policy (Nexto) on three fixed maps, so its claims about learning the game's dynamics rest on the assumption that it has captured the underlying physics rather than memorizing one policy's behavioral patterns.

What would settle it

If the model's rollouts degrade significantly when driven by action sequences that differ substantially from the Nexto bot's behavioral distribution — for example, sustained inaction, unusual aerial maneuvers, or adversarial inputs designed to push cars into states the bot never produces — then the model would have demonstrated pattern matching to one policy rather than learning general dynamics.

read the original abstract

We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model's physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 7 minor

Summary. This paper introduces MIRA, a 5B-parameter latent diffusion world model for four-player Rocket League that conditions on all players' simultaneous action streams, generates 20 fps in real time on a single B200 GPU, and remains stable over long horizons. The model predicts in the latent space of a representation autoencoder built on a frozen DINOv3-L feature extractor, trained with flow matching and diffusion forcing. The paper provides systematic ablations of the codec design (Tables 2–7, 21–25), the training objective (Table 8, Figures 9–11), the multiplayer conditioning scheme (Figure 14), and scaling behavior (Figure 16). It also introduces two targeted evaluation metrics: the Action Recoverability Ratio (ARR), validated against human judgment (Pearson r=0.84, Figure 12), and a game-state probe that reads physical quantities from the model's activations. The authors release their dataset, codebase, and a live demo.

Significance. This is a substantial contribution to interactive world modeling. The multiplayer conditioning design—tiling four views into a single grid with per-player action embeddings—is a clean and effective solution to multi-agent attribution. The systematic codec ablations (particularly Table 3 and Figure 7 showing that a frozen pretrained feature extractor is what prevents long-horizon drift, despite worse reconstruction) provide actionable design guidance for the field. The ARR metric and its validation against human preference studies is a valuable methodological contribution. The release of 10,000 hours of gameplay data with aligned physics state, full training/inference code, and a live interactive demo sets a high standard for reproducibility. The real-time inference system (Section 5) is well-engineered and documented.

major comments (2)
  1. §6.2, §6.8: The paper claims rollouts are 'both visually coherent and dynamically faithful to the commanded actions' and 'stay stable far beyond the training horizon' (abstract, §1, §6.1). However, the two lines of evidence are measured at different horizons: distributional metrics (gFID/gFVD/gFDD) extend to 300 seconds (Figures 7, 9), while the dynamical metrics—game-state probe error (Figure 16a) and ARR (Figure 13)—are reported only as functions of model size and training step, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This leaves open whether dynamical fidelity degrades over long horizons even as distributional quality remains flat. The claim of long-horizon stability would be substantially strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s). As word,
  2. §6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The paper acknowledges this limitation in §3.1 but does not revisit it when making the 'theory of mind' claim in §6.8. The claim should be qualified: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence for robustness to distribution shift, but does not directly test whether the unconditioned-player behavior generalizes beyond Nexto's strategy space.
minor comments (7)
  1. §4.2: The adaptive gradient-norm balancing rule is described as reusing 'the gradient-norm balancing that VQ-GAN applies to its single adversarial term,' but VQ-GAN balances between reconstruction and discriminator losses, not between two perceptual terms and reconstruction. The analogy could be stated more precisely.
  2. §6.3, Table 2: The pixel-space ARR is 'calibrated against real frames rather than a reconstruction' because pixel-space models have no codec. This makes the ARR comparison between latent and pixel space not apples-to-apples (the latent ARR divides by APrecon, the pixel ARR divides by APreal). The paper should note this asymmetry explicitly.
  3. §6.7, Figure 16a: The ball-position probe error is reported in 'Unreal units' but the axis label says '×10³' without specifying the unit in the caption. Adding the unit (uu) to the caption would improve clarity.
  4. §5: The paper states one full step takes 'roughly 70 ms end to end and produces two video frames (about 35 ms per frame),' which is within the 50 ms budget for 20 fps. It would be useful to report the variance or worst-case latency, since interactive applications are sensitive to tail latency, not just mean throughput.
  5. §6.9: The failure case of the ball moving on its own when untouched is attributed to data imbalance. It would strengthen the analysis to report how frequently this occurs quantitatively, similar to the uncommanded boost/jump counts.
  6. Figure 19 caption: The clock drift example shows the clock reading 4:54, 4:53, 4:53, 4:52, 4:54, 4:53 over five seconds. The text says 'it advances far too slowly and even ticks back up,' but the sequence also shows 4:52→4:54, which is a two-second jump forward followed by a one-second jump back. This pattern (both too slow and occasionally jumping) could be described more precisely.
  7. References: Several cited works have 2026 dates (e.g., Siméoni et al. 2025 for DINOv3 is listed as arXiv:2508.10104, but other references like Tong et al. 2026, Singh et al. 2026, Hansen-Estruch et al. 2026 appear to be from 2026). The mixing of 2025 and 2026 dates should be verified for consistency.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and the constructive feedback. Both major comments identify genuine gaps in our evaluation that we will address in revision. Below we respond point by point.

read point-by-point responses

  1. Referee: §6.2, §6.8: Distributional metrics extend to 300s but dynamical metrics (ARR, game-state probe) are not reported as functions of rollout horizon. The claim of long-horizon stability would be strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s).

    Authors: The referee is correct. Our distributional metrics (gFID/gFVD/gFDD) are tracked over rollout horizon up to 300 seconds (Figures 7, 9, 10), but our two dynamical metrics—ARR (Figure 13) and the game-state probe error (Figure 16a)—are reported only as functions of training step and model size, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This is a real gap: it leaves open the possibility that dynamical fidelity degrades over long horizons even as distributional quality remains flat, which would weaken our long-horizon stability claim. We will address this by computing both ARR and the game-state probe error at multiple rollout horizons (4s, 30s, 60s, 120s, 300s) on the flagship 5B model, conditioned on ground-truth actions, and adding the results as a new figure. We will also soften the abstract and §6.1 claims to specify that long-horizon stability has been verified for distributional quality and will be verified for dynamical fidelity in the revision. We note that the game-state probe is trained on real latents and applied to generated rollouts (§6.2), so extending it to multiple horizons is straightforward computationally; the main cost is running the probe over long rollouts, which we can do with existing infrastructure. revision: yes

  2. Referee: §6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The claim should be qualified.

    Authors: The referee raises a valid concern. We acknowledge in §3.1 that using a single bot policy (Nexto) limits behavioral diversity, but we do not revisit this limitation when making the 'theory of mind' claim in §6.8. The referee is right that, since all four cars in every training match are driven by independent instances of the same Nexto policy, the model's behavior for unconditioned players most plausibly reflects imitation of Nexto's policy distribution learned from pixels, rather than a general agent model that would transfer to arbitrary player strategies. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence—the model stays coherent under human control, which is outside the training action distribution—but this tests robustness to distribution shift in the conditioned player's actions, not whether the unconditioned-player behavior generalizes beyond Nexto's strategy space. We will revise §6.8 to qualify the claim explicitly: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. We will also add a forward reference to the §3.1 limitation at the point where the claim is made. We retain the observation that the model recovers complex decisions from pixels alone (the bot has access to privileged game state), but we will frame this as policy imitation from observation rather than 'theory of mind' in the general sense. revision: yes

Circularity Check

0 steps flagged

No circularity found: derivation chain is self-contained with external benchmarks

full rationale

The paper's derivation chain is self-contained. The codec builds on a frozen, externally-developed feature extractor (DINOv3-L, Siméoni et al. 2025) and the representation autoencoder paradigm (Zheng et al. 2025, Singh et al. 2026), both external. The world model uses flow matching (Lipman et al. 2023) and diffusion forcing (Chen et al. 2024), also external. Evaluation metrics are independently grounded: gFID/gFVD use Inception-V3 and a 3D backbone; gFDD and ARR use a frozen DINOv3-B probe calibrated on real held-out data (0.84 mAP, Table 20) and validated against human judgment (Pearson r=0.84, Spearman ρ=0.93, Figure 12). The game-state probe is trained on real latents and tested on generated rollouts—a legitimate transfer test, not a self-referential loop. Self-citations (GAIA-1, GAIA-2, Ramanana Rahary et al.) appear only in related work for context, not as load-bearing premises. The P-DINO perceptual loss in codec training and the FDD evaluation metric both use DINOv3 features, but they serve different purposes (reconstruction loss vs. distributional distance) and use different model variants (DINOv3-L for training, DINOv3-B for evaluation), so this is not circular. No step in the derivation reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

9 free parameters · 4 axioms · 2 invented entities

The free parameters are standard hyperparameters for a large-scale diffusion model, not ad hoc constants introduced to make a derivation work. The axioms are domain assumptions about the experimental setting (game as proxy, bot diversity, metric validity) rather than mathematical postulates. The invented entities (ARR, the codec) are empirically validated artifacts, not postulated physical objects.

free parameters (9)
  • Latent channels (C) = 32
    Chosen as the bottleneck dimensionality; not derived from a principle.
  • Context window (T) = 20 latent frames
    Fixed-size rolling window at inference; chosen for real-time constraints.
  • Perceptual loss weights (lambda_p, lambda_d) = Adaptively balanced
    Gradient-norm matched to L1 loss; the balancing rule itself is a design choice from VQ-GAN.
  • Action dropout rate = 0.1 per step, 0.5 subset-drop
    Probability of replacing a player's action embedding with an absent token during training.
  • Context noise std (inference) = 0.2
    Default re-noising level for past latents at inference; chosen as a light default that rescues the fragile distilled codec.
  • Aggregated DINOv3 blocks = {11, 13, 15, 17, 19, 21, 23}
    Hand-selected intermediate layers for feature aggregation.
  • Spatial/temporal downsampling = 2x2 spatial, 2x temporal
    Chosen compression factors for the bottleneck.
  • Training clip length = 80 frames (4s)
    Fixed training window length.
  • Flow-matching steps (inference) = 10
    Default sampling steps for evaluation.
axioms (4)
  • domain assumption Frozen DINOv3-L features provide a smooth, semantically meaningful latent space that prevents long-horizon rollout drift.
    Invoked in Section 4.2 and supported empirically in Table 3 and Figure 7, but presented as an assumption about why pretrained features help rather than a derived result.
  • domain assumption Rocket League is a useful proxy for real-world physically dynamic multi-agent environments.
    Stated in Section 1; the game is a video game simulation, not a real physical system, so transfer to real environments is assumed rather than demonstrated.
  • domain assumption The Action Recoverability Ratio (ARR) is a valid proxy for controllability.
    Section 6.2 defines ARR and Section 6.5 validates it against human judgment (r=0.84), but it remains a proxy that measures action visibility, not causal fidelity.
  • domain assumption Bot-generated gameplay (Nexto) provides sufficient behavioral diversity for training a general world model.
    Section 3.1 acknowledges this is a limitation; the model inherits one policy's style of play.
invented entities (2)
  • Action Recoverability Ratio (ARR) independent evidence
    purpose: Metric measuring how faithfully generated rollouts obey commanded actions, by checking if actions are recoverable from generated video via a probe.
    Validated against human preference (Figure 12, Pearson r=0.84, Spearman rho=0.93) and provides a falsifiable, quantitative handle on controllability.
  • Representation autoencoder codec with temporal downsampling independent evidence
    purpose: Compresses video into a compact latent at 10 Hz built on frozen DINOv3 features, for stable real-time world modeling.
    Ablated against pixel-space, from-scratch, and no-downsampling alternatives (Tables 2-5); the codec is a trained artifact, not a postulated entity.

pith-pipeline@v1.1.0-glm · 53391 in / 3072 out tokens · 476245 ms · 2026-07-07T15:24:37.032067+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

280 extracted references · 280 canonical work pages · 47 internal anchors

  1. [1]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Recurrent World Models Facilitate Policy Evolution , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  2. [2]

    International Conference on Machine Learning (ICML) , year =

    Learning Latent Dynamics for Planning from Pixels , author =. International Conference on Machine Learning (ICML) , year =

    work page

  3. [3]

    International Conference on Learning Representations (ICLR) , year =

    Dream to Control: Learning Behaviors by Latent Imagination , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  4. [4]

    International Conference on Learning Representations (ICLR) , year =

    Mastering Atari with Discrete World Models , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  5. [5]

    Nature , year =

    Mastering Diverse Domains through World Models , author =. Nature , year =

    work page

  6. [7]

    International Conference on Learning Representations (ICLR) , year =

    Transformers are Sample-Efficient World Models , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  7. [8]

    International Conference on Learning Representations (ICLR) , year =

    Transformer-based World Models Are Happy With 100k Interactions , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  8. [9]

    Zhang, Weipu and Wang, Gang and Sun, Jian and Yuan, Yetian and Huang, Gao , booktitle =

    work page

  9. [10]

    International Conference on Machine Learning (ICML) , year =

    Temporal Difference Learning for Model Predictive Control , author =. International Conference on Machine Learning (ICML) , year =

    work page

  10. [11]

    Hansen, Nicklas and Su, Hao and Wang, Xiaolong , booktitle =

    work page

  11. [12]

    2022 , note =

    A Path Towards Autonomous Machine Intelligence , author =. 2022 , note =

    work page 2022

  12. [13]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    work page

  13. [14]

    Transactions on Machine Learning Research (TMLR) , year =

    Revisiting Feature Prediction for Learning Visual Representations from Video , author =. Transactions on Machine Learning Research (TMLR) , year =

    work page

  14. [15]

    Assran, Mahmoud and others , journal =

    work page

  15. [16]

    Zhou, Gaoyue and Pan, Hengkai and LeCun, Yann and Pinto, Lerrel , booktitle =

    work page

  16. [17]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Video Diffusion Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  17. [18]

    and Poole, Ben and Norouzi, Mohammad and Fleet, David J

    Ho, Jonathan and Chan, William and Saharia, Chitwan and Whang, Jay and Gao, Ruiqi and Gritsenko, Alexey and Kingma, Diederik P. and Poole, Ben and Norouzi, Mohammad and Fleet, David J. and Salimans, Tim , journal =

    work page

  18. [19]

    Singer, Uriel and Polyak, Adam and Hayes, Thomas and Yin, Xi and An, Jie and Zhang, Songyang and Hu, Qiyuan and Yang, Harry and Ashual, Oron and Gafni, Oran and Parikh, Devi and Gupta, Sonal and Taigman, Yaniv , booktitle =

    work page

  19. [21]

    2024 , note =

    Video Generation Models as World Simulators , author =. 2024 , note =

    work page 2024

  20. [22]

    Yang, Zhuoyi and Teng, Jiayan and Zheng, Wendi and Ding, Ming and Huang, Shiyu and Xu, Jiazheng and Yang, Yuanming and Hong, Wenyi and Zhang, Xiaohan and Feng, Guanyu and others , booktitle =

    work page

  21. [23]

    Kong, Weijie and others , journal =

    work page

  22. [24]

    HaCohen, Yoav and others , journal =

    work page

  23. [25]

    International Conference on Learning Representations (ICLR) , year =

    Vision Transformers Need Registers , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  24. [26]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  25. [28]

    Shazeer, Noam , journal =

    work page

  26. [29]

    Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    Ainslie, Joshua and Lee-Thorp, James and de Jong, Michiel and Zemlyanskiy, Yury and Lebr. Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    work page

  27. [30]

    Findings of the Association for Computational Linguistics: EMNLP , year =

    Query-Key Normalization for Transformers , author =. Findings of the Association for Computational Linguistics: EMNLP , year =

    work page

  28. [31]

    International Conference on Machine Learning (ICML) , year =

    Scaling Vision Transformers to 22 Billion Parameters , author =. International Conference on Machine Learning (ICML) , year =

    work page

  29. [32]

    Su, Jianlin and Lu, Yu and Pan, Shengfeng and Murtadha, Ahmed and Wen, Bo and Liu, Yunfeng , journal =

    work page

  30. [33]

    International Conference on Machine Learning (ICML) , year =

    Is Space-Time Attention All You Need for Video Understanding? , author =. International Conference on Machine Learning (ICML) , year =

    work page

  31. [34]

    IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    Arnab, Anurag and Dehghani, Mostafa and Heigold, Georg and Sun, Chen and Lu. IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    work page

  32. [35]

    Polyak, Adam and others , journal =

    work page

  33. [36]

    Bar-Tal, Omer and Chefer, Hila and Tov, Omer and Herrmann, Charles and Paiss, Roni and Zada, Shiran and Ephrat, Ariel and Hur, Junhwa and Liu, Guanghui and Raj, Amit and others , booktitle =

    work page

  34. [37]

    International Conference on Machine Learning (ICML) , year =

    Kondratyuk, Dan and Yu, Lijun and Gu, Xiuye and Lezama, Jos. International Conference on Machine Learning (ICML) , year =

    work page

  35. [38]

    European Conference on Computer Vision (ECCV) , year =

    Photorealistic Video Generation with Diffusion Models , author =. European Conference on Computer Vision (ECCV) , year =

    work page

  36. [39]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    High-Resolution Image Synthesis with Latent Diffusion Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    work page

  37. [40]

    IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    Scalable Diffusion Models with Transformers , author =. IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    work page

  38. [41]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Denoising Diffusion Probabilistic Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  39. [42]

    International Conference on Learning Representations (ICLR) , year =

    Flow Matching for Generative Modeling , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  40. [43]

    International Conference on Learning Representations (ICLR) , year =

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  41. [44]

    and Boffi, Nicholas M

    Ma, Nanye and Goldstein, Mark and Albergo, Michael S. and Boffi, Nicholas M. and Vanden-Eijnden, Eric and Xie, Saining , booktitle =

    work page

  42. [45]

    International Conference on Learning Representations (ICLR) , year =

    Diffusion Models Are Real-Time Game Engines , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  43. [46]

    Bruce, Jake and Dennis, Michael and Edwards, Ashley and Parker-Holder, Jack and Shi, Yuge and Hughes, Edward and Lai, Matthew and Mavalankar, Aditi and Steigerwald, Richie and Apps, Chris and others , booktitle =

    work page

  44. [47]

    Diffusion for World Modeling: Visual Details Matter in

    Alonso, Eloi and Jelley, Adam and Micheli, Vincent and Kanervisto, Anssi and Storkey, Amos and Pearce, Tim and Fleuret, Fran. Diffusion for World Modeling: Visual Details Matter in. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  45. [48]

    Che, Haoxuan and He, Xuanhua and Liu, Quande and Jin, Cheng and Chen, Hao , booktitle =

    work page

  46. [49]

    Feng, Ruili and Zhang, Han and Yang, Zhantao and Xiao, Jie and Shu, Zhilei and Liu, Zhiheng and Zheng, Andy and Huang, Yukun and Liu, Yu and Zhang, Hongyang , journal =. The

    work page

  47. [50]

    Zhang, Yifan and Wei, Chunli and Wu, Dongdong and He, Xianglong and Xu, Yiqun and Lyu, Xinjie and Zhang, Yongchao and Liu, Hao and Chen, Yang and others , journal =

    work page

  48. [51]

    He, Xianglong and Zhang, Chunli and Wu, Dongdong and Zhang, Yifan and Xu, Yiqun and others , journal =

    work page

  49. [52]

    2026 , howpublished=

    Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory , author=. 2026 , howpublished=

    work page 2026

  50. [53]

    Guo, Junliang and Ye, Yang and He, Tianyu and Wu, Haoyu and Jiang, Yushu and Pearce, Tim and Bian, Jiang , journal =

    work page

  51. [54]

    Fan, Linxi and Wang, Guanzhi and Jiang, Yunfan and Mandlekar, Ajay and Yang, Yuncong and Zhu, Haoyi and Tang, Andrew and Huang, De-An and Zhu, Yuke and Anandkumar, Anima , booktitle =

    work page

  52. [55]

    Li, Jiaqi and Zhang, Junshu and Jiang, Boyuan and Wang, Yuxuan and Zhao, Yujie and others , journal =

    work page

  53. [56]

    Learning to Simulate Dynamic Environments with

    Kim, Seung Wook and Zhou, Yuhao and Philion, Jonah and Torralba, Antonio and Fidler, Sanja , booktitle =. Learning to Simulate Dynamic Environments with

    work page

  54. [57]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Playable Video Generation , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    work page

  55. [58]

    Position: Interactive Generative Video as Next-Generation Game Engine

    Position: Interactive Generative Video as Next-Generation Game Engine , author =. arXiv preprint arXiv:2503.17359 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  56. [59]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  57. [60]

    International Conference on Machine Learning (ICML) , year =

    History-Guided Video Diffusion , author =. International Conference on Machine Learning (ICML) , year =

    work page

  58. [61]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  59. [62]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    From Slow Bidirectional to Fast Autoregressive Video Diffusion Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    work page

  60. [63]

    International Conference on Learning Representations (ICLR) , year =

    One Step Diffusion via Shortcut Models , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  61. [64]

    International Conference on Machine Learning (ICML) , year =

    Consistency Models , author =. International Conference on Machine Learning (ICML) , year =

    work page

  62. [65]

    International Conference on Learning Representations (ICLR) , year =

    Improved Techniques for Training Consistency Models , author =. International Conference on Learning Representations (ICLR) , year =

    work page

  63. [68]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  64. [69]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Professor Forcing: A New Algorithm for Training Recurrent Networks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

    work page

  65. [70]

    International Conference on Machine Learning (ICML) , year =

    Rolling Diffusion Models , author =. International Conference on Machine Learning (ICML) , year =

    work page

  66. [71]

    Kim, Jihwan and Kang, Junoh and Choi, Jinyoung and Han, Bohyung , booktitle =

    work page

  67. [73]

    Nature , volume =

    World and Human Action Models towards Gameplay Ideation , author =. Nature , volume =

    work page

  68. [74]

    Pondaven, Alexander and Wu, Haoyu and Gilitschenski, Igor and Torr, Philip and Tulyakov, Sergey and Pizzati, Fabio and Siarohin, Aliaksandr , journal =

    work page

  69. [75]

    Liu, Fangfu and He, Kai and Ren, Xuanchi and others , journal =

    work page

  70. [76]

    Savva, Georgy and Michel, Oscar and Xie, Saining and others , journal =

    work page

  71. [77]

    Wu, Haoyu and Yu, Jiwen and Zou, Yingtian and Liu, Xihui , journal =

    work page

  72. [78]

    Po, Ryan and Zhang, Kai and Hertz, Amir and Wetzstein, Gordon and Wadhwa, Neal and Ruiz, Nataniel , journal =

    work page

  73. [79]

    Hu, Teng and others , journal =

    work page

  74. [80]

    Zhu, Shangwen and Peng, Yiran and Feng, Ruili and others , journal =

    work page

  75. [81]

    Liu, Andrew and Tucker, Richard and Jampani, Varun and Makadia, Ameesh and Snavely, Noah and Kanazawa, Angjoo , booktitle =

    work page

  76. [82]

    Li, Zhengqi and Wang, Qianqian and Snavely, Noah and Kanazawa, Angjoo , booktitle =

    work page

  77. [83]

    Liu, Ruoshi and Wu, Rundi and Van Hoorick, Basile and Tokmakov, Pavel and Zakharov, Sergey and Vondrick, Carl , booktitle =

    work page

  78. [84]

    Sargent, Kyle and Li, Zizhang and Shah, Tanmay and Herrmann, Charles and Yu, Hong-Xing and Zhang, Yunzhi and Chan, Eric Ryan and Lagun, Dmitry and Fei-Fei, Li and Sun, Deqing and Wu, Jiajun , booktitle =

    work page

  79. [85]

    Koh, Jing Yu and Lee, Honglak and Yang, Yinfei and Baldridge, Jason and Anderson, Peter , booktitle =

    work page

  80. [86]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Navigation World Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    work page

Showing first 80 references.