Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

RMIO: A Model-Based MARL Framework for Scenarios with Observation Loss in Some Agents

T0 review · 5 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RMIO reconstructs lost observations from a world model and corrects them with other agents' live data, keeping MARL policies stable when some agents see nothing.

desk verdict A genuinely new combination for a real MARL failure mode, with the main gaps being a missing loss-detector specification and a formal model that disagrees with the experimental protocol. read the letter →

arxiv 2411.19639 v1 pith:T3QDGSFL submitted 2024-11-29 cs.MA

Zifeng Shi , Meiqin Liu , Senlin Zhang , Ronghao Zheng , Shanling Dong This is my paper · ORCID
classification cs.MA
keywords multi-agentreinforcementlearningmodel-basedworldmodelobservationlosscorrectionblockcentralizedtrainingdecentralizedexecutionsampleefficiencystateestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most model-based multi-agent reinforcement learning assumes every agent receives a reliable observation stream at every step. This paper argues that a model-based framework can keep making good joint decisions when some agents receive no observation at all: a learned world model reconstructs the missing observations, and a correction block refines those reconstructions using the observations of agents that did see the environment. The framework also trains with centralized information but executes each agent from its own observations, so normal operation needs no continuous communication, and it adds reward smoothing plus a dual replay buffer to stabilise convergence. In SMAC and MaMuJoCo experiments, the paper reports that the method outperforms prior model-based and model-free baselines in both standard and observation-loss settings, with the correction block reducing reconstruction error by roughly one order of magnitude.

What carries the argument

The load-bearing mechanism is the pairing of a latent-variable world model with a correction block. The world model, built on a recurrent state-space model, provides a prior model that predicts the next latent state without the current observation and an observation predictor that decodes that latent state into a reconstructed observation; this gives an initial imputation for agents whose observations are missing. The correction block then takes the imputed observations for the $n-m$ lost agents together with the $m$ real observations, encodes both, mixes them with self-attention across agents, and decodes refined estimates, trained by an MSE loss that compares the refined estimates with the true observations. This two-stage impute-then-correct loop is what the paper credits for keeping joint decisions stable when some agents see nothing, and it is trained by masking real observations during training.

What would settle it

A direct test would run RMIO in the same benchmarks with a protocol in which all agents lose observations simultaneously at some steps, so $m=0$; the correction block's loss is undefined for $m=n$, and the paper neither defines this case nor gives an implementation of the detector that decides which agents lost observations. If the announced robustness at $p_{\text{loss}}=1$ relies on the random-subset protocol always leaving one observer and a perfect detector, those runs should degrade sharply.

Watch

Extended reading notes

Core claim

The central claim is that a multi-agent system can remain robust when a subset of agents is completely deprived of observations, by switching the world model from a data generator into an inference-time imputer. RMIO reconstructs the missing observations from the recurrent prior model and the observation predictor, then feeds the imputed values together with the true observations of the remaining agents into a correction block that is trained with an MSE loss to match the true observations. The corrected joint observation is what the policy consumes, and the policy is decoupled from the world model, giving centralized training with decentralized execution in normal settings and one-step communication only when a loss occurs. Across the SMAC and MaMuJoCo benchmarks, the paper reports consistently higher win rates and episode rewards than the compared baselines, and on the 3s_vs_3z map the correction loss falls from 0.007212 to 0.0009924.

Load-bearing premise

The method assumes that it can reliably tell which agents lost their observations and that at least one agent still receives a true observation at every step, because the correction block is only defined when at least one real observation exists.

Editorial extensions

If this is right

  • In normal environments, RMIO follows centralized training with decentralized execution, so agents act from their own local observations and do not need continuous communication; communication is triggered only when an observation is lost.
  • When observations are lost, the correction block shrinks reconstruction error substantially, and the paper reports that this preserves high win rates across SMAC maps even when the observation-loss probability reaches 1.
  • On hard, many-agent tasks, the paper reports diminishing effectiveness because agents cannot observe all teammates and the prior model's predictions become less accurate.
  • Reward smoothing and the dual experience replay buffer stabilise training against periods when the world model drifts, improving asymptotic convergence compared with earlier model-based multi-agent methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of the impute-then-correct loop suggests a broader design principle for model-based MARL: a dynamics model can be used at runtime as a missing-data imputer, not only as a source of pseudo-trajectories, which could transfer to sensor dropout in robot swarms and autonomous driving.
  • The paper's ploss=1 results are only meaningful if the random-subset protocol always leaves at least one agent with a real observation; the method's correction block is undefined when all agents lose observations simultaneously, so full simultaneous loss remains an open case.
  • A sharper test of the mechanism would isolate the correction block's effect on decisions: compare policies trained with and without correction at decision time while keeping the same world model, since the reported MSE reduction alone does not prove that corrected observations improve the chosen actions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes RMIO, a model-based multi-agent reinforcement learning (MARL) framework designed for Dec-POMDPs in which some agents may lose their observations entirely. RMIO trains a latent world model (RSSM-based) to predict and reconstruct missing observations, and a correction block that uses observations from agents that did not lose theirs to refine the imputed observations. The policy follows the CTDE paradigm in standard settings, with limited communication only when observation loss occurs. The method also incorporates reward smoothing, a dual-layer experience replay buffer, and an RNN-augmented policy. The authors evaluate RMIO on SMAC and MaMuJoCo benchmarks against MAMBA, MAG, MAPPO, QMIX, and FACMAC, in both standard settings and under varying observation-loss probabilities, and report that RMIO outperforms all baselines in both cases.

Significance. If the claims are substantiated, the paper makes a useful contribution: it is, to my knowledge, a plausible first model-based MARL approach that explicitly targets complete observation loss by combining prior-model imputation with inter-agent correction, while retaining CTDE execution and limiting communication to loss events. The correction-block ablation (RMIO vs RMIO*) is a sensible check, and the benchmark coverage across SMAC and MaMuJoCo is broad. However, the manuscript is not yet at a publishable standard: the formal definition of the loss process is inconsistent with the experimental protocol, the discriminator that drives the whole inference pipeline is unspecified, two theoretical claims are deferred to a missing appendix, and the observation-loss experiments are reported without the numerical precision needed to assess significance. The central idea is promising, but several load-bearing details need to be fixed before the claims can be accepted.

major comments (5)
  1. [§3 (Definition 1) vs §5.4.2, §5.4.3] The observation-loss protocol is defined inconsistently. Definition 1 gives each agent an independent per-agent loss probability p_i^t, so setting p_loss=1 in Section 5.4.3 would mean every agent loses its observation at every step (m=0). Under that reading, the correction block in Eq. (5) and Eq. (6) has no accurate observations to use as input, and Algorithm 2 line 12 calls f_cor with an empty reliable set, which is undefined. Section 5.4.2 instead describes p_loss in event-level terms ("a probability that results in a random subset of agents losing observation information") and, on four maps, fixes the number of lost agents to n-1 so that m=1. The paper must state which protocol is actually used, align Definition 1 with that protocol, and either handle the all-agents-lost case explicitly or restrict the claim to settings where m ≥ 1.
  2. [§4.3.2, Algorithm 2 line 3] The discriminator D is treated as an oracle that returns the set of agents with accurate observations, but no architecture, training procedure, or correctness assumption is provided. Since the paper's central scenario is that some agents "are completely unable to obtain any observational information," D determines the entire input to the correction block and the policy. Without a specification of D, the behavior of RMIO under imperfect detection, and especially in the all-agents-lost case, is undefined. This gap directly affects the claim that RMIO "ensures stable multi-agent decision-making" under observation loss.
  3. [§4.2.1, §4.2.2] The claims that reward smoothing preserves optimality and that the dual-layer experience replay buffer is effective are both deferred to an appendix that is not included in the arXiv manuscript. These are not merely presentational issues: the first claim is a formal statement about the policy objective, and the second is a justification for one of the proposed asymptotic-performance improvements. The authors should provide the proofs either in the main text or in a complete appendix.
  4. [§5.4.3, Figure 9] The observation-loss results are reported only as plots, with no visible error bars and no numeric standard deviations in the text; the caption says statistics are over 3 random seeds, but the figure appears to show only mean curves. Table 1 supplies numeric values with standard deviations for the standard environment, but no equivalent table is given for the observation-loss setting. As a result, the claim of "consistently outperforms" under varying p_loss cannot be quantitatively assessed. Please provide numeric results, error bars, or a table with means and standard deviations for all p_loss conditions, and consider reporting individual runs.
  5. [Algorithm 1 lines 25–35 vs §5.4.2] The training procedure for the correction block is underspecified: Algorithm 1 says only "mask partial agents' observation" without stating how many agents are masked, whether the number is fixed or random, or whether the masking distribution matches the test-time loss protocol. Section 5.4.2 fixes the number of lost agents to n-1 on four easy maps and uses a random subset on others. If the training-time masking distribution does not match the test-time protocol, the correction block is evaluated off-distribution. The masking distribution and any dependence on p_loss should be stated explicitly.
minor comments (7)
  1. [Throughout] Several typos and formatting inconsistencies need correction: "Related Workes" in the Section 2 heading, "StarCratII" in Section 5.1, "MAMUJOCO" in Figure 9, "ppiror" in Eq. (2), and "bo_t" / "bγt" in Eq. (4).
  2. [Eq. (5)] The notation in Eq. (5) has mismatched braces and is ambiguous: f_cor returns a set of corrected values, and the concatenation with the accurate observations should be defined more carefully, especially since the order of agents matters for later policy input.
  3. [§4.3.2] The sentence "implying that only m(m < n) observations {o_i_t}_{i=m+1}" is inconsistent: if m agents have accurate observations, the accurate set should be indexed {o_i_t}_{i=1}^m, not {i=m+1}. Please correct the indexing.
  4. [§5.4.3] The text refers to "Table ?? and Table ??" as illustrative examples, but these tables are missing from the manuscript. The placeholders must be filled before resubmission.
  5. [§4.2.1] The statement that smoothing "ensure[s] that the total rewards remain consistent" is imprecise: the finite-window Gaussian smoothing in Eq. (11) with boundary clipping does not exactly preserve the episode sum of rewards. The claim should be stated with the appropriate boundary caveat.
  6. [§5.4.3, Figure 9] The figure caption says standard deviations are part of the reported statistics, but the plots do not appear to contain shaded regions or error bars. Please clarify whether the curves include error bars and, if so, how they are shown.
  7. [§4.3.2, Figure 5] The "close" observation-loss mechanism that avoids re-communication by estimating other agents' actions from shared policy parameters is described informally. A short formal description (e.g., the exact information used and the update equations) would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: RMIO's observation-loss mechanism is benchmarked externally; the remaining gaps are completeness issues, not self-referential reasoning.

full rationale

The central derivation chain is not circular. RMIO trains a world model (Eq. 4) on real trajectories, trains a policy in the CTDE paradigm (Eqs. 7-10), and trains a correction block (Eq. 6) as a supervised imputer on masked observations. The correction block's target (the true masked observations) is used only in the loss; at execution the block is applied to genuinely missing observations, so the evaluation in Figures 7 and 9 against external baselines (MAMBA, MAG, MAPPO, QMIX, FACMAC) is not fitted into the input. There is no load-bearing self-citation chain: the method builds on MAMBA and Dreamer but benchmarks against them externally. I did find non-circular weaknesses that should be weighed separately: (1) Definition 1 defines per-agent independent loss but Section 5.4.2 describes event-level loss and fixes n-1 lost agents for four maps, while Section 5.4.3 reports ploss=1; under Definition 1 this gives m=0 where Eq. 5/6 and Algorithm 2 line 12 are undefined. (2) The discriminator D in Algorithm 2 line 3 is never specified. (3) The 'proofs' of reward smoothing and double replay buffer are relegated to an appendix that is not present. These are correctness/completeness concerns, not circularity, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 2 invented entities

The central method rests on standard RSSM and PPO machinery plus four paper-specific additions: reward smoothing, double replay buffer, correction block, and D. The first two are justified by proofs deferred to a missing appendix; the latter two are modules with no external validation. No fitted theoretical constants are involved, but several hyperparameters are unreported and the observation-loss protocol is not precisely defined.

free parameters (3)
  • reward smoothing window H and Gaussian sigma = not specified
    Equation (12) defines Gaussian smoothing weights with H and sigma; no values or sensitivity analysis are reported, yet the reward model and policy are trained on the smoothed rewards.
  • communication history length l = not specified, described as dynamically adjusted
    Algorithm 2 line 5 synchronizes observations and actions over l prior steps; l is never given and the 'close loss' criterion for skipping communication is described only qualitatively.
  • observation-loss duration and loss probability ploss = duration set to 10 steps; ploss swept over 0 to 1 in Figure 9
    These define the evaluation protocol; results are reported as a function of ploss, and the easy-map setting fixes n-1 lost agents, so performance claims are tied to this hand-chosen protocol.
assumptions (3)
  • ad hoc to paper Reward smoothing preserves the optimal policy.
    Section 4.2.1 asserts this and defers the proof to an appendix that is absent from the preprint; the claim is used to justify training on smoothed rewards.
  • domain assumption A discriminator D can perfectly identify which agents lost observations at each step.
    Algorithm 2 line 3 calls D(ot) with no implementation, training procedure, or error analysis; inference behavior depends on this detector being correct.
  • domain assumption The correction block can recover missing observations from the remaining agents' observations.
    Equation (5) and the MSE loss (6) assume inter-agent observations carry enough information; the paper itself notes in Section 5.4.3 that this fails on corridor-style tasks where agents cannot observe each other.
invented entities (2)
  • Correction block f_cor
    purpose: Refines world-model imputations of lost observations using accurate observations from other agents, as in Eq. 5.
    Introduced by this paper; evidence is only the internal MSE curve (Figure 8) and task performance, with no external falsifiable handle. It also has no defined behavior when m=0.
  • Observation-loss discriminator D
    purpose: Tells the algorithm which agents lost observations at execution time, as in Algorithm 2 line 3.
    Mentioned without architecture, training data, or accuracy analysis; its correctness is assumed for all reported deployment results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RMIO: A Model-Based MARL Framework for Scenarios with Observation Loss in Some Agents." pith.science (2026). https://pith.science/paper/T3QDGSFL

@misc{pith2026241119639,
  author       = {Pith},
  title        = {Pith review of: RMIO: A Model-Based MARL Framework for Scenarios with Observation Loss in Some Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T3QDGSFL}},
  note         = {Machine review of arXiv:2411.19639}
}
read the original abstract

In recent years, model-based reinforcement learning (MBRL) has emerged as a solution to address sample complexity in multi-agent reinforcement learning (MARL) by modeling agent-environment dynamics to improve sample efficiency. However, most MBRL methods assume complete and continuous observations from each agent during the inference stage, which can be overly idealistic in practical applications. A novel model-based MARL approach called RMIO is introduced to address this limitation, specifically designed for scenarios where observation is lost in some agent. RMIO leverages the world model to reconstruct missing observations, and further reduces reconstruction errors through inter-agent information integration to ensure stable multi-agent decision-making. Secondly, unlike CTCE methods such as MAMBA, RMIO adopts the CTDE paradigm in standard environment, and enabling limited communication only when agents lack observation data, thereby reducing reliance on communication. Additionally, RMIO improves asymptotic performance through strategies such as reward smoothing, a dual-layer experience replay buffer, and an RNN-augmented policy model, surpassing previous work. Our experiments conducted in both the SMAC and MaMuJoCo environments demonstrate that RMIO outperforms current state-of-the-art approaches in terms of asymptotic convergence performance and policy robustness, both in standard mission settings and in scenarios involving observation loss.

Figures

Figures reproduced from arXiv: 2411.19639 by the authors.

Figure 1
Figure 1. At each step a random subset of agents has a probability of experiencing observation loss. The process of completing and correcting [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Network composition and inference process of correction block. The scenario in the figure assumes that there are [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Double Experience Replay Buffer structure. In such unstable scenarios, a dual experience replay buffer structure is designed in RMIO to mitigate this issue. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The whole reasoning process of RMIO facing observation loss(only [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Taking agent 3 as an example, the light colored parts in the figure represent the status information of other agents that agent 3 cannot [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Comparisons with other baselines. The solid line represents the running average of 3 di [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 7
Figure 7. Figure 7: Compared to other CTCE baselines, RMIO, as a CTDE method, achieves a significantly higher win rate [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Comparison of loss function values before and after correcting the world model predictions. Before/ [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: The performance of RMIO method and its ablation control group, as well as other model-based methods, in 1000 random matches under [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Agent Reinforcement Learning in Wireless Distributed Networks for 6G

    cs.IT 2025-02 conditional novelty 1.0 of 10

    A comprehensive survey of multi-agent reinforcement learning for wireless distributed networks in 6G, covering structures, algorithms, enhanced techniques, and applications.

Reference graph

Works this paper leans on

40 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Matignon, L

    L. Matignon, L. Jeanpierre, A.-I. Mouaddib, Coordinated multi-robot exploration under communication constraints using decentralized markov decision processes, Proceedings of the AAAI Conference on Artificial Intelligence (2022) 2017–2023

    work page 2022

  2. [2]

    S.-M. Hung, S. N. Givigi, A q-learning approach to flocking with uavs in a stochastic environment, IEEE Transactions on Cybernetics (2017) 186–197

    work page 2017

  3. [3]

    C. You, J. Lu, D. Filev, P. Tsiotras, Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning, Robotics and Autonomous Systems (2019) 1–18

    work page 2019

  4. [4]

    Shalev-Shwartz, S

    S. Shalev-Shwartz, S. Shammah, A. Shashua, Safe, multi-agent, reinforcement learning for autonomous driving, arXiv preprint arXiv:1610.03295 (Oct 2016)

    arXiv 2016

  5. [5]

    Rashid, M

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, S. Whiteson, Monotonic value function factorisation for deep multi-agent reinforcement learning, Journal of Machine Learning Research 21 (178) (2020) 1–51

    work page 2020

  6. [6]

    Baker, I

    B. Baker, I. Kanitscheider, T. Markov, Y . Wu, G. Powell, B. McGrew, I. Mordatch, Emergent tool use from multi-agent autocurricula, International Conference on Learning Representations (2020)

    work page 2020

  7. [7]

    D. Ye, Z. Liu, M. Sun, B. Shi, P. Zhao, H. Wu, H. Yu, S. Yang, X. Wu, Q. Guo, Q. Chen, Y . Yin, H. Zhang, T. Shi, L. Wang, Q. Fu, W. Yang, L. Huang, Mastering complex control in moba games with deep reinforcement learning, Proceedings of the AAAI Conference on Artificial Intelligence 34 (04) (2020) 6672–6679

    work page 2020

  8. [8]

    Gronauer, K

    S. Gronauer, K. Diepold, Multi-agent deep reinforcement learning: a survey, Artificial Intelligence Review (2022) 895–943. 15

    work page 2022

Show all 40 references
  1. [9]

    Hafner, T

    D. Hafner, T. Lillicrap, J. Ba, M. Norouzi, Dream to control: learning behaviors by latent imagination, International Conference on Learning Representations (2020)

    2020

  2. [10]

    Jänner, J

    M. Jänner, J. Fu, M. Zhang, S. Levine, When to trust your model: model-based policy optimization, Advances in Neural Information Processing Systems (Jun 2019)

    2019

  3. [11]

    T. M. Moerland, J. Broekens, A. Plaat, C. M. Jonker, et al., Model-based reinforcement learning: a survey, Foundations and Trends ® in Machine Learning 16 (1) (2023) 1–118

    2023

  4. [12]

    Krupnik, I

    O. Krupnik, I. Mordatch, A. Tamar, Multi-agent reinforcement learning with multi-step generative models, Conference on Robot Learning (2020) 776–790

    2020

  5. [13]

    Egorov, A

    V . Egorov, A. Shpilman, Scalable multi-agent model-based reinforcement learning, Proceedings of the International Conference on Au- tonomous Agents and Multiagent Systems (2022) 381–390

    2022

  6. [14]

    Z. Wu, C. Yu, C. Chen, J. Hao, H. H. Zhuo, Models as agents: optimizing multi-step predictions of interactive local models in model- based multi-agent reinforcement learning, Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conferen...

    2023

  7. [15]

    F. A. Oliehoek, C. Amato, A concise introduction to decentralized pomdps, SpringerBriefs in Intelligent Systems (Jan 2016)

    2016

  8. [16]

    Y . Sun, W. Li, D. Zhao, Convergence time and speed of multi-agent systems in noisy environments, Chaos: An Interdisciplinary Journal of Nonlinear Science 22 (4) (Dec 2012)

    2012

  9. [17]

    B. Chen, M. Xu, Z. Liu, L. Li, D. Zhao, Delay-aware multi-agent reinforcement learning for cooperative and competitive environments, arXiv preprint arXiv:2005.05441 (2020)

    2020 arXiv

  10. [18]

    Matignon, L

    L. Matignon, L. Jeanpierre, A.-I. Mouaddib, Coordinated multi-robot exploration under communication constraints using decentralized markov decision processes, Proceedings of the AAAI Conference on Artificial Intelligence (2012) 2017–2023

    2012

  11. [19]

    J. Gao, S. Wang, X. Wang, Y . Zhang, X. Yang, Reinforcement learning for multi-agent with asynchronous missing information fusion method, International Journal of Machine Learning and Cybernetics (2024) 1–17

    2024

  12. [20]

    Samvelyan, T

    M. Samvelyan, T. Rashid, C. Schroeder de Witt, G. Farquhar, N. Nardelli, T. G. Rudner, C.-M. Hung, P. H. Torr, J. Foerster, S. Whiteson, The starcraft multi-agent challenge, Proceedings of the International Conference on Autonomous Agents and MultiAgent Systems (2019) 2186–2188

    2019

  13. [21]

    B. Peng, T. Rashid, C. Schroeder de Witt, P.-A. Kamienny, P. Torr, W. Böhmer, S. Whiteson, Facmac: Factored multi-agent centralised policy gradients, Advances in Neural Information Processing Systems 34 (2021) 12208–12221

    2021

  14. [22]

    Feinberg, A

    V . Feinberg, A. Wan, I. Stoica, M. I. Jordan, J. E. Gonzalez, S. Levine, Model-based value estimation for e fficient model-free reinforcement learning, arXiv preprint arXiv:1803.00101 (2018)

    2018 arXiv

  15. [23]

    Ayoub, Z

    A. Ayoub, Z. Jia, C. Szepesvari, M. Wang, L. Yang, Model-based reinforcement learning with value-targeted regression, International Con- ference on Machine Learning (2020) 463–474

    2020

  16. [24]

    Hafner, T

    D. Hafner, T. Lillicrap, M. Norouzi, J. Ba, Mastering atari with discrete world models, International Conference on Learning Representations (2021)

    2021

  17. [25]

    Hafner, J

    D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap, Mastering diverse domains through world models, arXiv preprint arXiv:2301.04104 (2023)

    2023 arXiv

  18. [26]

    Micheli, E

    V . Micheli, E. Alonso, F. Fleuret, Transformers are sample-e fficient world models, International Conference on Learning Representations (2023)

    2023

  19. [27]

    Zhang, G

    W. Zhang, G. Wang, J. Sun, Y . Yuan, G. Huang, Storm: E fficient stochastic transformer based world models for reinforcement learning, Advances in Neural Information Processing Systems 36 (2024)

    2024

  20. [28]

    Robine, M

    J. Robine, M. Höftmann, T. Uelwer, S. Harmeling, Transformer-based world models are happy with 100k interactions, Deep Reinforcement Learning Workshop NeurIPS 2022 (2022)

    2022

  21. [29]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, Advances in Neural Information Processing Systems (Jun 2017)

    2017

  22. [30]

    M. G. Bellemare, Y . Naddaf, J. Veness, M. Bowling, The arcade learning environment: an evaluation platform for general agents, Journal of Artificial Intelligence Research (2018) 253–279

    2018

  23. [31]

    J. Wang, Y . Liu, B. Li, Proceedings of the AAAI Conference on Artificial Intelligence (04) (2020) 6202–6209

    2020

  24. [32]

    K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y . Bengio, Learning phrase representations using rnn encoder-decoder for statistical machine translation, Proceedings of the Conference on Empirical Methods in Natural Language Processing (Jan 2014)

    2014

  25. [33]

    D. P. Kingma, M. Welling, Auto-encoding variational bayes, CoRR abs /1312.6114 (2013)

    2013 arXiv

  26. [34]

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, Y . Wu, The surprising e ffectiveness of ppo in cooperative multi-agent games, Advances in Neural Information Processing Systems 35 (2022) 24611–24624

    2022

  27. [35]

    V . Lee, P. Abbeel, Y . Lee, Dreamsmooth: Improving model-based reinforcement learning via reward smoothing, International Conference on Learning Representations (2024)

    2024

  28. [36]

    Hafner, Benchmarking the spectrum of agent capabilities, Deep Reinforcement Learning Workshop NeurIPS 2021 (2021)

    D. Hafner, Benchmarking the spectrum of agent capabilities, Deep Reinforcement Learning Workshop NeurIPS 2021 (2021)

    2021

  29. [37]

    Kannan, D

    H. Kannan, D. Hafner, C. Finn, D. Erhan, Robodesk: A multi-task reinforcement learning benchmark, https://github.com/ google-research/robodesk (2021)

    2021

  30. [38]

    Plappert, M

    M. Plappert, M. Andrychowicz, A. Ray, B. McGrew, B. Baker, G. Powell, J. Schneider, J. Tobin, M. Chociej, P. Welinder, et al., Multi-goal reinforcement learning: Challenging robotics environments and request for research, arXiv preprint arXiv:1802.09464 (2018)

    2018 arXiv

  31. [39]

    S. Wang, X. Zhan, Y . Zhai, J. Shen, H. Wang, Performance estimation for kalman filter based multi-agent cooperative navigation by employing graph theory, Aerospace Science and Technology 112 (2021) 106628

    2021

  32. [40]

    J. Li, S. Tang, J. Guo, Event-triggered distributed cooperative extended kalman filter based on formation estimation, Aerospace Science and Technology 138 (2023) 108326. 16

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.