REVIEW 8 cited by
Diffusion for World Modeling: Visual Details Matter in Atari
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
World models constitute a promising approach for training reinforcement learning agents in a safe and sample-efficient manner. Recent world models predominantly operate on sequences of discrete latent variables to model environment dynamics. However, this compression into a compact discrete representation may ignore visual details that are important for reinforcement learning. Concurrently, diffusion models have become a dominant approach for image generation, challenging well-established methods modeling discrete latents. Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a Model Of eNvironment Dreams), a reinforcement learning agent trained in a diffusion world model. We analyze the key design choices that are required to make diffusion suitable for world modeling, and demonstrate how improved visual details can lead to improved agent performance. DIAMOND achieves a mean human normalized score of 1.46 on the competitive Atari 100k benchmark; a new best for agents trained entirely within a world model. We further demonstrate that DIAMOND's diffusion world model can stand alone as an interactive neural game engine by training on static Counter-Strike: Global Offensive gameplay. To foster future research on diffusion for world modeling, we release our code, agents, videos and playable world models at https://diamond-wm.github.io.
Forward citations
Cited by 8 Pith papers
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning
BDPO computes the behavior-regularization penalty for diffusion policies as a sum of per-denoising-step KL divergences and optimizes with a two-time-scale actor-critic, achieving strong D4RL performance.
-
Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregression
HMA is a masked autoregressive transformer that predicts future video and actions across many robot embodiments, running up to 15x faster than prior diffusion-based video simulators while matching or improving visual ...
-
Quo Vadis, World Modeling?
An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.
-
Value Flows
Value Flows fits the full return distribution in RL with a flow-matching critic and reweights its learning objective by estimated return variance; the central theoretical guarantee does not follow from the stated equations.
-
Exploratory Diffusion Model for Unsupervised Reinforcement Learning
A diffusion-model denoising loss serves as an intrinsic reward to guide unsupervised RL exploration, plus an alternating fine-tuning scheme for diffusion policies.
-
Pre-Trained Video Generative Models as World Simulators
A lightweight action-conditioning module and a motion-reinforced loss convert pre-trained video generators into action-following world simulators that also speed up model-based reinforcement learning.
-
Humans Coexist, So Must Embodied Artificial Agents
Coexistence, defined as sustained meaningful and reciprocal interaction among an agent, humans, and environment, is presented as a necessary design goal for embodied AI.
Discussion (0). Continue with ORCID to comment.