Pith. sign in

REVIEW 11 cited by

TransDreamer: Reinforcement Learning with Transformer World Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2202.09481 v2 pith:E6UHICXD submitted 2022-02-19 cs.LG

Chang Chen , Yi-Fu Wu , Jaesik Yoon , Sungjin Ahn This is my paper · ORCID
classification cs.LG
keywords modelworldagentmbrltransformertransformer-baseddreamerlearning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The Dreamer agent provides various benefits of Model-Based Reinforcement Learning (MBRL) such as sample efficiency, reusable knowledge, and safe planning. However, its world model and policy networks inherit the limitations of recurrent neural networks and thus an important question is how an MBRL framework can benefit from the recent advances of transformers and what the challenges are in doing so. In this paper, we propose a transformer-based MBRL agent, called TransDreamer. We first introduce the Transformer State-Space Model, a world model that leverages a transformer for dynamics predictions. We then share this world model with a transformer-based policy network and obtain stability in training a transformer-based RL agent. In experiments, we apply the proposed model to 2D visual RL and 3D first-person visual RL tasks both requiring long-range memory access for memory-based reasoning. We show that the proposed model outperforms Dreamer in these complex tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.

  2. Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents

    cs.AI 2025-12 unverdicted novelty 6.0 of 10

    Argos is an agentic verifier that adaptively picks scoring functions to evaluate accuracy, localization, and reasoning quality, enabling stronger multimodal RL training for AI agents.

  3. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  4. RynnVLA-002: A Unified Vision-Language-Action and World Model

    cs.RO 2025-11 conditional novelty 5.0 of 10

    A single model that jointly predicts robot actions and future images outperforms separate action-only and video-only models on LIBERO and real SO100 manipulation tasks.

  5. TransZero: Parallel Tree Expansion in MuZero using Transformer Networks

    cs.LG 2025-09 conditional novelty 4.0 of 10

    TransZero parallelizes Monte Carlo tree search expansion using a transformer dynamics network and a variance-based evaluator, achieving up to an 11x wall-clock speedup over MuZero without sacrificing final reward.

  6. Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challenges

    cs.LG 2025-08 conditional novelty 4.0 of 10

    A survey reviewing how world models and agentic AI could be combined to give edge devices predictive, proactive decision-making, with a taxonomy of methods, applications, and challenges.

  7. Reinforcement Learning in hyperbolic space for multi-step reasoning

    cs.LG 2025-07 reject novelty 4.0 of 10

    Hyperbolic transformer policies are claimed to beat vanilla transformer policies by 32-45% on a handful of reasoning and control problems, but the evidence is too weak to support the claim.

  8. A Survey: Learning Embodied Intelligence from Physical Simulators and World Models

    cs.RO 2025-07 conditional novelty 4.0 of 10

    Embodied intelligence learning is reviewed through the complementary lenses of physical simulators and world models, with a proposed IR-L0 to IR-L4 robot capability taxonomy.

  9. TransDreamerV3: Implanting Transformer In DreamerV3

    cs.LG 2025-06 reject novelty 4.0 of 10

    TransDreamerV3 replaces DreamerV3's GRU memory with a simple transformer encoder and reports mixed gains on four Atari and Crafter tasks.

  10. Reinforcement Learning: From Algorithms To Foundation Models

    cs.AI 2026-07 conditional novelty 3.0 of 10

    A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.

  11. From 2D to 3D Cognition: A Brief Survey of General World Models

    cs.CV 2025-06 conditional novelty 3.0 of 10

    A survey proposing a two-pillar, three-capability framework that organizes recent AI world models by their transition from 2D visual prediction to 3D cognition.

Pith tools