Pith. sign in

REVIEW 4 major objections 6 minor 32 references

GAWM: Global-Aware World Model for Multi-Agent Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GAWM claims that a Transformer-based observation-fusion world model makes model-based MARL outperform model-free and prior model-based methods on all eight SMAC maps tested.

desk verdict Useful architectural idea in model-based MARL, but the near-zero baseline win rates on easy SMAC maps make the superiority claim unsubstantiated until the comparisons are reproduced. read the letter →

arxiv 2501.10116 v1 pith:7C6DGSD7 submitted 2025-01-17 cs.MA

Zifeng Shi , Meiqin Liu , Senlin Zhang , Ronghao Zheng , Shanling Dong , Ping Wei This is my paper · ORCID
classification cs.MA
keywords worldmodelmulti-agentreinforcementlearningsampleefficiencyglobalstaterepresentationobservationfusionrewardtrendmodelingcentralizedtrainingdecentralizedexecutionSMAC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that model-based multi-agent reinforcement learning has been held back by world models that reconstruct each agent's state from its own partial observation, so the generated pseudo-trajectories disagree about the global reward, discount, and next-observation signals. It proposes GAWM, a world model that fuses all agents' local observations through a Transformer before predicting latent states, and it claims this one architectural change makes the generated data globally consistent and stable enough for a CTDE policy to converge well. The authors report that on all eight StarCraft Multi-Agent Challenge maps they test, GAWM achieves the highest final win rate in a fixed low number of environment steps, beating model-based baselines MAMBA and MAG as well as model-free MAPPO and QMIX. The reason this matters is that sample efficiency without sacrificing final performance is the main barrier to deploying MARL in real systems where interaction data is expensive.

What carries the argument

The load-bearing object is the observation-fusion module: a Transformer that takes each agent's local observation $o^i_t$ together with the recurrent hidden states $h_t$ and produces a fused embedding $g^i_t$ that the posterior model uses to sample the latent state $z^i_t$. This replaces the decentralized state reconstruction used by prior world models, in which each agent's latent state is built from its own partial observation alone. The world model is otherwise a recurrent state-space model with action fusion, prior and posterior latent paths, and predictors for observations, team reward, and discount factor, trained with the evidence-lower-bound-style loss of Eq. (3). Two auxiliary mechanisms carry the stability claim: reward trend modeling (Eqs. (4)-(5)), which trains the reward predictor on Gaussian-smoothed team rewards instead of exact values, and a double experience replay buffer that separates real from pseudo trajectories. The policy is a decoupled MAPPO-style actor that consumes raw local observations, so the world model is not part of the execution-time policy.

What would settle it

On a map like corridor or 3s vs 5z, retrain GAWM with the smoothing window set to $H=0$ so the reward model sees exact rewards, keeping obs-fusion and everything else fixed; if the win-rate gap over MAMBA and MAG largely disappears, the claimed advantage is carried by reward smoothing rather than by the global-aware state representation. The paper reports no such ablation.

Watch

Extended reading notes

Core claim

GAWM claims to be the first multi-agent world model that restores global consistency to pseudo-sample generation while staying within the centralized-training-decentralized-execution (CTDE) paradigm. Its central discovery is that fusing the local observations of all agents with a Transformer before the posterior model, rather than decoding each agent's latent state from its own observation, gives the world model a globally coherent representation of the shared environment, and that this coherence is what stabilizes online world-model learning and lets the policy converge. The paper also reports that replacing exact team-reward prediction with Gaussian-smoothed reward trend modeling, and keeping real and pseudo trajectories in separate replay buffers, further stabilizes training. On the eight SMAC maps from easy to super hard, the method reports the best average win rate against MAMBA, MAG, MAPPO, and QMIX, with markedly smaller variance across seeds; offline metrics GCI and GPE confirm lower cross-agent inconsistency and lower prediction error than the baselines.

Load-bearing premise

The paper assumes that training the reward predictor on Gaussian-smoothed team rewards leaves the optimal policy unchanged, and it gives no proof; if temporal smoothing distorts the reward signal, the reported win rates could reflect a biased reward proxy rather than a better world model.

Editorial extensions

If this is right

  • GAWM reports the highest average win rate on all eight SMAC maps in the fixed low-sample budget, including 93(3) on 2s vs 1sc versus 86(4) for MAG and 64(15) for MAMBA.
  • The ablation shows that removing obs-fusion increases world-model loss fluctuations and destabilizes the win-rate curve, supporting the claim that global observation fusion is the cause of the stability gain.
  • The offline GCI and GPE results indicate the world model's predictions are more globally consistent and more accurate than MAG's and MAMBA's on the tested maps.
  • Because the policy consumes only local observations, the method can be executed in the decentralized fashion required by CTDE, unlike the CTCE baselines.
  • Reward trend modeling suggests that exact reward reconstruction is not needed for convergence, so world models can be trained with a lower modeling burden.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: varying the smoothing window $H$ and $\sigma$ in Eqs. (4)-(5) while holding obs-fusion fixed would separate how much of the reported win rate comes from global representation versus from reward smoothing; the paper reports no such sweep.
  • The GCI metric defines global consistency by agreement among agents' reconstructed global states; a natural extension the authors do not pursue is using GCI as a training signal or early-stopping criterion for the world model.
  • The paper's argument implies that other partial-observability sources of inconsistency, such as non-uniform observation ranges, could be addressed by the same fusion-before-posterior pattern in settings beyond StarCraft, such as multi-robot or autonomous-driving domains.
  • One could also ask whether the benefit persists when the number of agents grows large, since a single Transformer fusing all agents' observations scales quadratically in agent count; the paper tests at most eight agents.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GAWM, a model-based multi-agent reinforcement learning (MARL) method that augments a latent-variable world model with a Transformer-based observation-fusion module, models team rewards via temporal smoothing, and decouples the world model from the policy network to adhere to the centralized-training decentralized-execution (CTDE) paradigm. The authors claim that this design improves global-state representation consistency, stabilizes pseudo-sample generation, and, as a result, reaches state-of-the-art performance on the SMAC benchmark, uniformly outperforming model-based (MAMBA, MAG) and model-free (MAPPO, QMIX) baselines across eight maps. They also introduce two offline metrics, the Global Consistency Index (GCI) and Global Prediction Error (GPE), to support the claim that GAWM produces more consistent and accurate world-model predictions.

Significance. If the empirical results hold, GAWM would be a meaningful advance for sample-efficient model-based MARL, specifically by addressing the decentralized-reconstruction limitation of prior world models and by enabling a CTDE-compatible policy that does not depend on the world model's latent state. The paper's strengths are its clear articulation of a real weakness in existing multi-agent world models, a concrete and reasonably motivated architectural fix (obs-fusion), and an attempt to isolate the source of the gain via ablations and offline prediction-quality metrics. However, the significance is conditional: the central outperformance claim rests on baseline numbers that are not credible as reported, and the reward-smoothing step, which is central to the method, is justified only by an unproven optimality assertion. The paper also provides no code, no hyperparameters, and only three seeds, so the current evidence is insufficient to support the strength of the claimed 'consistently outperforms' conclusion.

major comments (4)
  1. [Table 1] The baseline numbers in Table 1 are implausible for standard implementations of MAPPO and QMIX on SMAC. Reporting 0.0% win rates for MAPPO and QMIX on 2s vs 1sc after 15k environment steps, on 3s vs 3z after 50k steps, and on 3s vs 4z and corridor after hundreds of thousands of steps is far below what established public implementations of these algorithms achieve on these maps under comparable or smaller budgets. For example, MAPPO commonly exceeds 50% on 2s vs 1sc well before 100k steps, and QMIX solves it within a few hundred thousand steps. A 0% average over 1000 test episodes across all three seeds therefore strongly suggests a mismatch in environment wrappers, reward/termination settings, training budgets, or hyperparameter choices rather than an honest sample-efficiency comparison. Since the paper's central claim is that GAWM 'consistently outperforms the existing methods,' this issue is load-bearing. The authors should either provide the full experimental setup and code for all baselines, or rerun the baselines with verified configurations (e.g., public SMAC-compatible implementations) and show that the reported numbers are reproducible. Without this, the 0% baselines cannot be taken at face value.
  2. [Section 3.1.1, Eq. (4)-(5)] The claim that 'using smoothed rewards in MARL does not compromise strategy optimality' is asserted without proof or a concrete citation. DreamSmooth (Ref. [30]) is a single-agent method, and its theoretical justification does not automatically transfer to cooperative MARL with a shared team reward and decentralized policies. The reward smoothing is applied before training the reward predictor, and the policy is then trained on imagined trajectories with these smoothed predicted rewards. If the smoothing biases the return ordering of policies, the reported win-rate improvements could be an artifact of a distorted reward proxy rather than a genuine benefit of global-aware world-model representation. This is load-bearing because reward trend modeling is one of the three stated contributions. The authors should provide a formal argument (e.g., a reward-shaping equivalence or a proof that the optimal joint policy is invariant to the smoothing operation), or at minimum an ablation that trains GAWM with unsmoothed rewards while keeping all other components identical and shows that the observed gains are not caused by the smoothing itself.
  3. [Section 4.1] The statistical evidence is too thin for the claim that GAWM 'consistently outperforms' baselines on all eight maps. Only three random seeds are used, no significance tests are reported, and the paper does not state whether the parenthetical values in Table 1 are standard deviations across seeds, standard errors, or something else. Even where the averages favor GAWM, the small number of seeds makes the 'consistently' claim fragile, especially on maps where the differences between GAWM and the best baseline are moderate (e.g., 8m, where GAWM is 90(2) vs. MAG's 63(8), and MAMBA's 37(7)). That difference is large, but without seed-level data or a test (e.g., a paired test across the three seeds), the paper's claim of consistently superior convergence and stability is not substantiated. Recommend increasing the number of seeds, reporting per-seed results, and performing a simple significance test or at least reporting confidence intervals.
  4. [Section 2.3 / Table 1] MACD [17], which the paper itself describes as a relevant CTDE model-based MARL method, is conspicuously absent from the baseline comparisons. Given that GAWM's novelty includes being a CTDE method and that the paper criticizes prior model-based methods for being CTCE or for requiring per-agent world models, a comparison against MACD is directly relevant to the positioning of the contribution. Adding MACD results, or providing a reason for its omission, would strengthen the empirical claims. Without it, the claim that GAWM outperforms 'existing methods' in the CTDE setting is incomplete.
minor comments (6)
  1. [Table 1 caption] The caption mentions 'episode reward (in MaMuJoCo)' but Table 1 contains only SMAC maps and win-rate metrics; either add MaMuJoCo experiments or remove the reference to MaMuJoCo from the caption.
  2. [Section 4.3.1, Eq. (7)] The GCI metric in Eq. (7) is partly circular: it measures consistency of per-agent predicted global states, rewards, and discount factors, which is exactly the property that the obs-fusion module is designed to improve. This makes the GCI ablation informative about the internal mechanism but not an independent validation of the method's overall quality. The accompanying win-rate ablation in Fig. 5 is the more convincing evidence; please clarify in the text that GCI is an interpretability/diagnostic metric rather than an independent performance measure.
  3. [Ablation Studies / Fig. 4] The caption of Fig. 4, which reports the world-model training loss, uses the label 'win_rate' on the y-axis; the axis label should be 'loss' to match the described content.
  4. [Section 4.3.2] The experimental design states that 'each method generates 1000 pairs of pseudo trajectory segments and real trajectory segments,' but the pairing criterion is not specified; please state how a pseudo segment is matched to a real segment (e.g., same initial state, same action sequence, or random pairing).
  5. [General] The paper does not provide hyperparameters for the world model, the Transformer fusion modules, the policy optimization, the smoothing window H and sigma in Eq. (5), the KL weight beta in Eq. (3), or the thresholds epsilon_r and epsilon_gamma in Eq. (7). These are necessary for reproducibility; please include them in a table or appendix.
  6. [General] There are several typos and formatting artifacts: 'ppiror' in Eq. (1e) should be 'pprior'; 'M ost' at the start of Section 3.1.2 is a spacing artifact; 'e fficiency' appears in multiple places; and the keyword list contains 'Feature Represetation' instead of 'Representation'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central outperformance claim is evaluated on the external SMAC benchmark, and the design components are supported by external citations rather than self-referential definitions.

full rationale

The paper's central derivation is self-contained. GAWM's architecture (Eqs. 1-2), world-model loss (Eq. 3), reward-smoothing procedure (Eqs. 4-5), and policy update are standard MBRL/PPO components with external antecedents (Dreamer-style RSSM; MAPPO; DreamSmooth for reward smoothing). The main claim—that GAWM 'consistently outperforms the existing methods'—is established by win rates on the external SMAC benchmark (Table 1, Fig. 3), not by a fitted parameter or a metric that encodes the method's design. The GCI and GPE metrics in Section 4.3 are evaluation instruments defined independently of the training loss; although they measure consistency and prediction error—objectives GAWM was designed to improve—they still require empirical estimation over rollouts and are compared across baselines, so they do not reduce to the method's definitions. No load-bearing self-citation was found: the relevant prior works cited (MAMBA, MAG, MACD, DreamSmooth, MAPPO) are not by the present authors, and no uniqueness theorem is imported from the authors' own work. The reward-trend modeling assertion in Section 3.1.1 that 'using smoothed rewards in MARL does not compromise strategy optimality' is an unproved optimality assumption, which is a correctness risk, not a circularity. Likewise, the implausible near-zero model-free baselines in Table 1 are an empirical-reproducibility concern and fall outside circularity analysis under the rule that non-standard-consensus claims are not circularity arguments.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim depends on several unstated hyperparameters and on assumptions about how rewards and latent representations behave; none of these are fitted to the SMAC data in the paper, but they are not provided either, which weakens reproducibility.

free parameters (4)
  • Reward smoothing window H
    Eq. (4) uses H to define the smoothing interval; the value is not reported.
  • Reward smoothing standard deviation sigma
    Eq. (5) uses sigma to define Gaussian weights; the value is not reported.
  • KL loss weight beta
    Eq. (3) weights the KL divergence by beta; no value is given.
  • GCI thresholds epsilon_r and epsilon_gamma
    Eq. (7) uses thresholds for reward and discount consistency; the values are not stated.
assumptions (3)
  • ad hoc to paper Reward smoothing preserves the optimal policy in cooperative MARL.
    Stated in Sec. 3.1.1 after Eq. (5): 'Importantly, using smoothed rewards in MARL does not compromise strategy optimality.' No proof is given; this bridges the smoothed reward model and true environment objective, and if false, GAWM's reported win rates may not reflect performance on the true reward.
  • domain assumption The latent state z_t contains sufficient information from all agents for policy learning.
    The world model is trained with reconstruction and KL losses; the policy is trained on pseudo-rollouts, assuming that latent representations are an adequate substitute for the true state in a Dec-POMDP. Standard in RSSM-based methods but unproven here.
  • domain assumption Attention fusion of local observations produces a globally consistent representation.
    The paper assumes that cross-agent attention (obs-fusion) yields a latent state that better reflects the global state; no formal argument is given, only empirical comparisons.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GAWM: Global-Aware World Model for Multi-Agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/7C6DGSD7

@misc{pith2026250110116,
  author       = {Pith},
  title        = {Pith review of: GAWM: Global-Aware World Model for Multi-Agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7C6DGSD7}},
  note         = {Machine review of arXiv:2501.10116}
}
read the original abstract

In recent years, Model-based Multi-Agent Reinforcement Learning (MARL) has demonstrated significant advantages over model-free methods in terms of sample efficiency by using independent environment dynamics world models for data sample augmentation. However, without considering the limited sample size, these methods still lag behind model-free methods in terms of final convergence performance and stability. This is primarily due to the world model's insufficient and unstable representation of global states in partially observable environments. This limitation hampers the ability to ensure global consistency in the data samples and results in a time-varying and unstable distribution mismatch between the pseudo data samples generated by the world model and the real samples. This issue becomes particularly pronounced in more complex multi-agent environments. To address this challenge, we propose a model-based MARL method called GAWM, which enhances the centralized world model's ability to achieve globally unified and accurate representation of state information while adhering to the CTDE paradigm. GAWM uniquely leverages an additional Transformer architecture to fuse local observation information from different agents, thereby improving its ability to extract and represent global state information. This enhancement not only improves sample efficiency but also enhances training stability, leading to superior convergence performance, particularly in complex and challenging multi-agent environments. This advancement enables model-based methods to be effectively applied to more complex multi-agent environments. Experimental results demonstrate that GAWM outperforms various model-free and model-based approaches, achieving exceptional performance in the challenging domains of SMAC.

Figures

Figures reproduced from arXiv: 2501.10116 by the authors.

Figure 1
Figure 1. The current mainstream world models adopt a centralized state [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Dual Experience Replay Buffer structure. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparisons with other baselines. The solid line represents the running average of 3 di [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Training loss curve for the world model. The solid line represents the running average of 3 di [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Win rate curve for ablation experiments. The solid line represents the running average of 3 di [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 25 canonical work pages

  1. [30]

    V . Lee, P. Abbeel, Y . Lee, Dreamsmooth: Improving model-based rein- forcement learning via reward smoothing, International Conference on Learning Representations (2024)

    work page 2024

  2. [17]

    Venugopal, S

    A. Venugopal, S. Milani, F. Fang, B. Ravindran, Mabl: Bi-level latent- variable world model for sample-e fficient multi-agent reinforcement learning, in: Proceedings of the 23rd International Conference on Au- tonomous Agents and Multiagent Systems, AAMAS ’24, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2024, p. 1865–1873

    work page 2024

  3. [1]

    Rashid, M

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, S. Whiteson, Monotonic value function factorisation for deep multi-agent reinforcement learning, Journal of Machine Learning Research 21 (178) (2020) 1–51. URL http://jmlr.org/papers/v21/20-081.html

    work page 2020

  4. [2]

    Baker, I

    B. Baker, I. Kanitscheider, T. Markov, Y . Wu, G. Powell, B. McGrew, I. Mordatch, Emergent tool use from multi-agent autocurricula, Interna- tional Conference on Learning Representations (2020)

    work page 2020

  5. [3]

    D. Ye, Z. Liu, M. Sun, B. Shi, P. Zhao, H. Wu, H. Yu, S. Yang, X. Wu, Q. Guo, Q. Chen, Y . Yin, H. Zhang, T. Shi, L. Wang, Q. Fu, W. Yang, L. Huang, Mastering complex control in MOBA games with deep rein- forcement learning, Proceedings of the AAAI Conference on Artificial Intelligence 34 (4) (2020) 6672–6679. doi:https://doi.org/10. 1609/aaai.v34i04.6144

    work page 2020

  6. [4]

    Matignon, L

    L. Matignon, L. Jeanpierre, A.-I. Mouaddib, Coordinated multi-robot ex- ploration under communication constraints using decentralized Markov decision processes, Proceedings of the AAAI Conference on Artificial Intelligence 26 (2022) 2017–2023. doi:https://doi.org/10.1609/ aaai.v26i1.8380

    work page 2022

  7. [5]

    S.-M. Hung, S. N. Givigi, A Q-learning approach to flocking with UA Vs in a stochastic environment, IEEE Transactions on Cybernet- ics 47 (1) (2017) 186–197. doi:https://doi.org/10.1109/tcyb. 2015.2509646

    arXiv 2017

  8. [6]

    M. T. Ramezanlou, H. Schwartz, I. Lambadaris, M. Barbeau, Enhancing cooperative multi-agent reinforcement learning through the integration of R-STDP and federated learning, Neurocomputing 617 (2025) 129005. doi:https://doi.org/10.1016/j.neucom.2024.129005

    work page arXiv 2025

Show all 32 references
  1. [7]

    C. You, J. Lu, D. Filev, P. Tsiotras, Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning, Robotics and Autonomous Systems 114 (2019) 1–18. doi: https://doi.org/10.1016/j.robot.2019.01.003

    2019 doi

  2. [8]

    Shalev-Shwartz, S

    S. Shalev-Shwartz, S. Shammah, A. Shashua, Safe, multi-agent, reinforcement learning for autonomous driving, arXiv preprint arXiv:1610.03295 (Oct 2016)

    2016 arXiv

  3. [9]

    H. Gao, M. Zhao, X. Zheng, C. Wang, L. Zhou, Y . Wang, L. Ma, B. Cheng, Z. Wu, Y . Li, An improved hierarchical deep reinforce- ment learning algorithm for multi-intelligent vehicle lane change, Neu- rocomputing 609 (2024) 128482. doi:https://doi.org/10.1016/ j.neucom.2024.128482

    2024

  4. [10]

    Hafner, T

    D. Hafner, T. Lillicrap, J. Ba, M. Norouzi, Dream to control: learning behaviors by latent imagination, International Conference on Learning Representations (2020)

    2020

  5. [11]

    J ¨anner, J

    M. J ¨anner, J. Fu, M. Zhang, S. Levine, When to trust your model: model- based policy optimization, Advances in Neural Information Processing Systems (Jun 2019)

    2019

  6. [12]

    T. M. Moerland, J. Broekens, A. Plaat, C. M. Jonker, et al., Model-based reinforcement learning: a survey, V ol. 16, Now Publishers, Inc., 2023

    2023

  7. [13]

    Malekzadeh, M

    P. Malekzadeh, M. Hou, K. N. Plataniotis, Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learning, Neurocomputing 530 (2023) 165–187. doi:https://doi. org/10.1016/j.neucom.2023.01.076

    2023 doi

  8. [14]

    Krupnik, I

    O. Krupnik, I. Mordatch, A. Tamar, Multi-agent reinforcement learning with multi-step generative models, Conference on Robot Learning (2020) 776–790

    2020

  9. [15]

    Egorov, A

    V . Egorov, A. Shpilman, Scalable multi-agent model-based reinforcement learning, Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (2022) 381–390

    2022

  10. [16]

    Z. Wu, C. Yu, C. Chen, J. Hao, H. H. Zhuo, Models as agents: Optimizing multi-step predictions of interactive local models in model-based multi- agent reinforcement learning, Proceedings of the AAAI Conference on Artificial Intelligence 37 (9) (2023) 10435–10443. doi:https://d...

    2023 doi

  11. [18]

    Samvelyan, T

    M. Samvelyan, T. Rashid, C. Schroeder de Witt, G. Farquhar, N. Nardelli, T. G. Rudner, C.-M. Hung, P. H. Torr, J. Foerster, S. Whiteson, The Star- Craft multi-agent challenge, Proceedings of the International Conference on Autonomous Agents and MultiAgent Systems (2019) 2186–2188

    2019

  12. [19]

    F. A. Oliehoek, C. Amato, A concise introduction to decentralized POMDPs, Springer Cham, 2016. doi:https://doi.org/10.1007/ 978-3-319-28929-8

    2016

  13. [20]

    Feinberg, A

    V . Feinberg, A. Wan, I. Stoica, M. I. Jordan, J. E. Gonzalez, S. Levine, Model-based value estimation for e fficient model-free reinforcement learning, arXiv preprint arXiv:1803.00101 (2018)

    2018 arXiv

  14. [21]

    Ayoub, Z

    A. Ayoub, Z. Jia, C. Szepesvari, M. Wang, L. Yang, Model-based rein- forcement learning with value-targeted regression, International Confer- ence on Machine Learning (2020) 463–474

    2020

  15. [22]

    Hafner, T

    D. Hafner, T. Lillicrap, M. Norouzi, J. Ba, Mastering Atari with dis- crete world models, International Conference on Learning Representa- tions (2021)

    2021

  16. [23]

    Hafner, J

    D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap, Mastering diverse domains through world models, arXiv preprint arXiv:2301.04104 (2023)

    2023 arXiv

  17. [24]

    Micheli, E

    V . Micheli, E. Alonso, F. Fleuret, Transformers are sample-efficient world models, International Conference on Learning Representations (2023)

    2023

  18. [25]

    Zhang, G

    W. Zhang, G. Wang, J. Sun, Y . Yuan, G. Huang, Storm: Efficient stochas- tic transformer based world models for reinforcement learning, Advances in Neural Information Processing Systems 36 (2024)

    2024

  19. [26]

    Robine, M

    J. Robine, M. H ¨oftmann, T. Uelwer, S. Harmeling, Transformer-based world models are happy with 100k interactions, International Conference on Learning Representations (2023)

    2023

  20. [27]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, Advances in Neural Information Processing Systems (Jun 2017)

    2017

  21. [28]

    K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y . Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing (2014) 17...

    2014 doi

  22. [29]

    D. P. Kingma, Auto-encoding variational bayes, arXiv preprint arXiv:1312.6114 (2013)

    2013 arXiv

  23. [31]

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, Y . Wu, The sur- prising effectiveness of PPO in cooperative multi-agent games, Advances in Neural Information Processing Systems 35 (2022) 24611–24624

    2022

  24. [32]

    J. Wang, Y . Liu, B. Li, Reinforcement learning with perturbed rewards, Proceedings of the AAAI Conference on Artificial Intelligence 04 (2020) 6202–6209. doi:https://doi.org/10.1609/aaai.v34i04.6086. 9

    2020 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.