{"id":"0b84e330-efbe-4e55-90ed-c590a10066c5","arxiv_id":"2411.19639","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RMIO is a model-based MARL framework that reconstructs lost agent observations with a world model, corrects them using other agents' observations, and reports higher performance than prior methods in SMAC and MaMuJoCo.","lead":"RMIO is a new model-based multi-agent reinforcement learning framework that keeps teams of agents working when some agents lose their sensor observations; it fills in missing observations with a learned world model and corrects those guesses using information from agents that can still see.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Definition 1 (per-agent loss) contradicts the ploss=1 experiments: at ploss=1, m=0 and the correction block Eq. 5/6 is undefined, so the central observation-loss claim is unsubstantiated at its reported extreme.","rationale":"After reading the paper in good faith, I find the empirical comparisons and the architecture plausible, but the formalization has a boundary inconsistency that the paper itself introduces. Definition 1 is the only formal model of observation loss, and it is per-agent. The method's correction module, its central novelty, is mathematically defined only when the number of agents with accurate observations m is at least 1. The experiments claim success at ploss=1, but under Definition 1 that is exactly the m=0 case. This is not a missing hyperparameter or a minor implementation detail; it is a condition under which the proposed inference procedure is not specified. The reader's verdict already flags the m≥1 requirement and the unspecified discriminator; my analysis sharpens this into an internal inconsistency between Eq. 1 and Section 5.4.3. If the authors intended ploss as an event-level probability over a random subset of agents, they should revise Definition 1 and state that the subset size is always between 1 and n−1; if they intended per-agent loss, they must extend the correction block to m=0 (e.g., by using only the prior-model imputation) and re-run the ploss=1 experiments. The absence of code and the placeholder tables in Section 5.4.3 make the ambiguity unresolvable from the preprint alone. Because the central claim is about robustness to observation loss, this boundary case is exactly where the contribution must be well-defined. I therefore agree with the reader's conditional verdict: the paper should be accepted only after the loss protocol and the m=0 handling are specified and validated. No change from the reader's CONDITIONAL verdict is needed.","tokens_in":15894,"tokens_out":6985,"duration_ms":59427,"concrete_test":"Run the 3s_vs_3z SMAC observation-loss experiment under Definition 1 with per-agent ploss=1 and an Oracle D (or read the code if released). At every step m=0; if Algorithm 2 cannot produce an action for the correction block or the reported win rate collapses, the Section 5.4.3 claim is invalid for the formal setting. Alternatively, re-run with event-level loss (one random subset of size < n loses) and confirm that ploss=1 leaves m≥1; if so, update Definition 1 and the text to state that at least one agent always retains an observation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RMIO's formal problem statement (Definition 1, Eq. 1) defines each agent's observation as independently present with probability p_t^i and absent with 1-p_t^i. Under this model, setting ploss=1 (as in Section 5.4.3) means every agent loses its observation at every step, i.e., m=0. But the correction block is defined only for 1 ≤ m < n: Eq. 5 requires the set {o_t^i}_{i=1}^m of accurate observations, and the training loss Eq. 6 computes MSE only over the n−m reconstructed entries, yet the correction block f_cor has no accurate input when m=0. Algorithm 2 line 3, 'm<n' branch, then calls f_cor(∅, {hat o_i}_{i=1}^n), which is not specified. Section 5.4.2 instead describes ploss as 'a probability that results in a random subset of agents losing observation information' (event-level), and for four maps fixes the number of lost agents to n−1, making m=1. Thus the reported 'even when ploss=1' performance in Section 5.4.3 is only meaningful under the event-level protocol, contradicting Definition 1. If ploss is per-agent, the extreme case is undefined; if event-level, the formal model must be revised. The discriminator D is also unspecified: Algorithm 2 line 3 treats D as an oracle returning the reliable subset, but no architecture, training, or assumption is given, so the system's behavior when an agent truly receives no signal is unknown. These gaps directly affect the central claim that RMIO 'ensures stable multi-agent decision-making' when 'some agents are completely unable to obtain any observational information'.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RMIO, a model-based multi-agent reinforcement learning (MARL) framework designed for Dec-POMDPs in which some agents may lose their observations entirely. RMIO trains a latent world model (RSSM-based) to predict and reconstruct missing observations, and a correction block that uses observations from agents that did not lose theirs to refine the imputed observations. The policy follows the CTDE paradigm in standard settings, with limited communication only when observation loss occurs. The method also incorporates reward smoothing, a dual-layer experience replay buffer, and an RNN-augmented policy. The authors evaluate RMIO on SMAC and MaMuJoCo benchmarks against MAMBA, MAG, MAPPO, QMIX, and FACMAC, in both standard settings and under varying observation-loss probabilities, and report that RMIO outperforms all baselines in both cases.","tokens_in":16182,"tokens_out":6899,"duration_ms":62026,"significance":"If the claims are substantiated, the paper makes a useful contribution: it is, to my knowledge, a plausible first model-based MARL approach that explicitly targets complete observation loss by combining prior-model imputation with inter-agent correction, while retaining CTDE execution and limiting communication to loss events. The correction-block ablation (RMIO vs RMIO*) is a sensible check, and the benchmark coverage across SMAC and MaMuJoCo is broad. However, the manuscript is not yet at a publishable standard: the formal definition of the loss process is inconsistent with the experimental protocol, the discriminator that drives the whole inference pipeline is unspecified, two theoretical claims are deferred to a missing appendix, and the observation-loss experiments are reported without the numerical precision needed to assess significance. The central idea is promising, but several load-bearing details need to be fixed before the claims can be accepted.","major_comments":[{"comment":"The observation-loss protocol is defined inconsistently. Definition 1 gives each agent an independent per-agent loss probability p_i^t, so setting p_loss=1 in Section 5.4.3 would mean every agent loses its observation at every step (m=0). Under that reading, the correction block in Eq. (5) and Eq. (6) has no accurate observations to use as input, and Algorithm 2 line 12 calls f_cor with an empty reliable set, which is undefined. Section 5.4.2 instead describes p_loss in event-level terms (\"a probability that results in a random subset of agents losing observation information\") and, on four maps, fixes the number of lost agents to n-1 so that m=1. The paper must state which protocol is actually used, align Definition 1 with that protocol, and either handle the all-agents-lost case explicitly or restrict the claim to settings where m ≥ 1.","section":"§3 (Definition 1) vs §5.4.2, §5.4.3"},{"comment":"The discriminator D is treated as an oracle that returns the set of agents with accurate observations, but no architecture, training procedure, or correctness assumption is provided. Since the paper's central scenario is that some agents \"are completely unable to obtain any observational information,\" D determines the entire input to the correction block and the policy. Without a specification of D, the behavior of RMIO under imperfect detection, and especially in the all-agents-lost case, is undefined. This gap directly affects the claim that RMIO \"ensures stable multi-agent decision-making\" under observation loss.","section":"§4.3.2, Algorithm 2 line 3"},{"comment":"The claims that reward smoothing preserves optimality and that the dual-layer experience replay buffer is effective are both deferred to an appendix that is not included in the arXiv manuscript. These are not merely presentational issues: the first claim is a formal statement about the policy objective, and the second is a justification for one of the proposed asymptotic-performance improvements. The authors should provide the proofs either in the main text or in a complete appendix.","section":"§4.2.1, §4.2.2"},{"comment":"The observation-loss results are reported only as plots, with no visible error bars and no numeric standard deviations in the text; the caption says statistics are over 3 random seeds, but the figure appears to show only mean curves. Table 1 supplies numeric values with standard deviations for the standard environment, but no equivalent table is given for the observation-loss setting. As a result, the claim of \"consistently outperforms\" under varying p_loss cannot be quantitatively assessed. Please provide numeric results, error bars, or a table with means and standard deviations for all p_loss conditions, and consider reporting individual runs.","section":"§5.4.3, Figure 9"},{"comment":"The training procedure for the correction block is underspecified: Algorithm 1 says only \"mask partial agents' observation\" without stating how many agents are masked, whether the number is fixed or random, or whether the masking distribution matches the test-time loss protocol. Section 5.4.2 fixes the number of lost agents to n-1 on four easy maps and uses a random subset on others. If the training-time masking distribution does not match the test-time protocol, the correction block is evaluated off-distribution. The masking distribution and any dependence on p_loss should be stated explicitly.","section":"Algorithm 1 lines 25–35 vs §5.4.2"}],"minor_comments":[{"comment":"Several typos and formatting inconsistencies need correction: \"Related Workes\" in the Section 2 heading, \"StarCratII\" in Section 5.1, \"MAMUJOCO\" in Figure 9, \"ppiror\" in Eq. (2), and \"bo_t\" / \"bγt\" in Eq. (4).","section":"Throughout"},{"comment":"The notation in Eq. (5) has mismatched braces and is ambiguous: f_cor returns a set of corrected values, and the concatenation with the accurate observations should be defined more carefully, especially since the order of agents matters for later policy input.","section":"Eq. (5)"},{"comment":"The sentence \"implying that only m(m < n) observations {o_i_t}_{i=m+1}\" is inconsistent: if m agents have accurate observations, the accurate set should be indexed {o_i_t}_{i=1}^m, not {i=m+1}. Please correct the indexing.","section":"§4.3.2"},{"comment":"The text refers to \"Table ?? and Table ??\" as illustrative examples, but these tables are missing from the manuscript. The placeholders must be filled before resubmission.","section":"§5.4.3"},{"comment":"The statement that smoothing \"ensure[s] that the total rewards remain consistent\" is imprecise: the finite-window Gaussian smoothing in Eq. (11) with boundary clipping does not exactly preserve the episode sum of rewards. The claim should be stated with the appropriate boundary caveat.","section":"§4.2.1"},{"comment":"The figure caption says standard deviations are part of the reported statistics, but the plots do not appear to contain shaded regions or error bars. Please clarify whether the curves include error bars and, if so, how they are shown.","section":"§5.4.3, Figure 9"},{"comment":"The \"close\" observation-loss mechanism that avoids re-communication by estimating other agents' actions from shared policy parameters is described informally. A short formal description (e.g., the exact information used and the update equations) would improve reproducibility.","section":"§4.3.2, Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready for publication in its current form. The main obstacle is not the general idea but the inconsistency between the formal model (Definition 1) and the experimental protocol, and the missing specification of the discriminator D and the correction-block training distribution. The standard-environment results are suggestive, but the observation-loss results—the paper's headline claim—need to be reproducible and quantitatively reported. If the authors can clarify the protocol, add the missing proofs and tables, and provide code or detailed hyperparameters, the paper could become a solid contribution. I would also suggest that the authors explicitly discuss the m=0 case and either handle it in the algorithm or state it as a limitation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RMIO is a sensible, well-scoped empirical paper. It targets a real failure mode in cooperative MARL—agents that completely lose their observations—and shows that a world model can fill in the missing data, then a correction block using the observations of other agents can reduce the reconstruction error. The combination of an RSSM-based world model, the masked-training correction block, and CTDE-style execution with communication only on loss is genuinely new relative to MAMBA and MAG, and the experiments on SMAC and MaMuJoCo are broad and consistently in RMIO's favor. The gains over strong model-based baselines are plausible and the ablation showing an order-of-magnitude MSE reduction from the correction block is concrete. Citation pattern is appropriate; MAMBA and MAG are the right comparators.\n\nThe soft spots are mostly gaps in evidence and specification rather than internal contradictions. The stress-test note about ploss = 1 does not quite land. The experimental protocol in Section 5.4.2 is event-level: a probability that a random subset of agents loses, and for four easy maps the number of lost agents is fixed to n−1. Under that protocol, ploss = 1 does not imply m = 0, so the correction block still has reliable inputs. What is true is that the formal Definition 1 is per-agent, which conflicts with the experimental protocol, and the m = 0 case is genuinely undefined. So the formal model needs revision and the paper should state that the all-agents-lost case is out of scope. The discriminator D in Algorithm 2 is a real gap: it is treated as an oracle that returns the reliable subset, but no architecture, training procedure, or assumption is given, and the central claim depends on it. The deferred appendix proofs for reward smoothing and the double buffer, the 'Table ??' placeholders, only 3 seeds, and no code release also make the empirical content harder to check. These are fixable weaknesses, not fatal ones.\n\nWho this is for: anyone working on robust MARL, model-based RL under degraded observations, or sensor-failure handling in multi-agent systems. It is not a field-reshaping theoretical result, but it is a useful mechanism with credible empirical support. I would accept it for peer review rather than desk reject, but I would push for a revised formal model, a specification of the loss detector, the missing appendix proofs, and ideally a code release before publication.","headline":"A genuinely new combination for a real MARL failure mode, with the main gaps being a missing loss-detector specification and a formal model that disagrees with the experimental protocol.","tokens_in":695,"tokens_out":1483,"would_cite":false,"duration_ms":40954,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RMIO reconstructs lost observations from a world model and corrects them with other agents' live data, keeping MARL policies stable when some agents see nothing.","keywords":["multi-agent reinforcement learning","model-based reinforcement learning","world model","observation loss","correction block","centralized training decentralized execution","sample efficiency","state estimation"],"falsifier":"A direct test would run RMIO in the same benchmarks with a protocol in which all agents lose observations simultaneously at some steps, so $m=0$; the correction block's loss is undefined for $m=n$, and the paper neither defines this case nor gives an implementation of the detector that decides which agents lost observations. If the announced robustness at $p_{\\text{loss}}=1$ relies on the random-subset protocol always leaving one observer and a perfect detector, those runs should degrade sharply.","tokens_in":15611,"feed_emoji":"🤖","tokens_out":7569,"duration_ms":62100,"temperature":0.7,"pith_summary":"Most model-based multi-agent reinforcement learning assumes every agent receives a reliable observation stream at every step. This paper argues that a model-based framework can keep making good joint decisions when some agents receive no observation at all: a learned world model reconstructs the missing observations, and a correction block refines those reconstructions using the observations of agents that did see the environment. The framework also trains with centralized information but executes each agent from its own observations, so normal operation needs no continuous communication, and it adds reward smoothing plus a dual replay buffer to stabilise convergence. In SMAC and MaMuJoCo experiments, the paper reports that the method outperforms prior model-based and model-free baselines in both standard and observation-loss settings, with the correction block reducing reconstruction error by roughly one order of magnitude.","feed_headline":"MARL policies stay stable when some agents observe nothing","feed_subtitle":"A model-based framework reconstructs lost observations, then refines them with other agents' live data.","key_machinery":"The load-bearing mechanism is the pairing of a latent-variable world model with a correction block. The world model, built on a recurrent state-space model, provides a prior model that predicts the next latent state without the current observation and an observation predictor that decodes that latent state into a reconstructed observation; this gives an initial imputation for agents whose observations are missing. The correction block then takes the imputed observations for the $n-m$ lost agents together with the $m$ real observations, encodes both, mixes them with self-attention across agents, and decodes refined estimates, trained by an MSE loss that compares the refined estimates with the true observations. This two-stage impute-then-correct loop is what the paper credits for keeping joint decisions stable when some agents see nothing, and it is trained by masking real observations during training.","core_discovery":"The central claim is that a multi-agent system can remain robust when a subset of agents is completely deprived of observations, by switching the world model from a data generator into an inference-time imputer. RMIO reconstructs the missing observations from the recurrent prior model and the observation predictor, then feeds the imputed values together with the true observations of the remaining agents into a correction block that is trained with an MSE loss to match the true observations. The corrected joint observation is what the policy consumes, and the policy is decoupled from the world model, giving centralized training with decentralized execution in normal settings and one-step communication only when a loss occurs. Across the SMAC and MaMuJoCo benchmarks, the paper reports consistently higher win rates and episode rewards than the compared baselines, and on the 3s_vs_3z map the correction loss falls from 0.007212 to 0.0009924.","pith_inferences":["The success of the impute-then-correct loop suggests a broader design principle for model-based MARL: a dynamics model can be used at runtime as a missing-data imputer, not only as a source of pseudo-trajectories, which could transfer to sensor dropout in robot swarms and autonomous driving.","The paper's ploss=1 results are only meaningful if the random-subset protocol always leaves at least one agent with a real observation; the method's correction block is undefined when all agents lose observations simultaneously, so full simultaneous loss remains an open case.","A sharper test of the mechanism would isolate the correction block's effect on decisions: compare policies trained with and without correction at decision time while keeping the same world model, since the reported MSE reduction alone does not prove that corrected observations improve the chosen actions."],"forward_implications":["In normal environments, RMIO follows centralized training with decentralized execution, so agents act from their own local observations and do not need continuous communication; communication is triggered only when an observation is lost.","When observations are lost, the correction block shrinks reconstruction error substantially, and the paper reports that this preserves high win rates across SMAC maps even when the observation-loss probability reaches 1.","On hard, many-agent tasks, the paper reports diminishing effectiveness because agents cannot observe all teammates and the prior model's predictions become less accurate.","Reward smoothing and the dual experience replay buffer stabilise training against periods when the world model drifts, improving asymptotic convergence compared with earlier model-based multi-agent methods."],"supporting_citations":[{"why":"Supplies the multi-agent recurrent world model and the CTCE baseline that RMIO adapts and compares against.","marker":"[13]"},{"why":"Provides a model-based MARL baseline that treats local models as decision-making agents and is used in standard and observation-loss comparisons.","marker":"[14]"},{"why":"Supplies the policy-gradient algorithm that RMIO adopts as its actor-critic policy model.","marker":"[34]"},{"why":"Provides the SMAC benchmark and its maps used for the discrete cooperative-control experiments.","marker":"[20]"},{"why":"Provides the MaMuJoCo benchmark and one of the compared model-free baselines for continuous-control experiments.","marker":"[21]"},{"why":"Supplies the discrete latent world-model design that the multi-agent predecessor is built on and that RMIO adapts.","marker":"[24]"},{"why":"Supplies the reward-smoothing approach that RMIO adopts to make reward modeling more stable.","marker":"[35]"},{"why":"Supplies the Transformer attention machinery used in the communication block and the correction block.","marker":"[29]"}],"fun_headline_variants":["RMIO keeps MARL agents steady when observations drop out","Model-based MARL fills missing observations for robust teamwork","MARL with lost observations? RMIO rebuilds them from others","RMIO: world model reconstructs lost agent views on the fly","For MARL, RMIO turns missing observations into no problem"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that it can reliably tell which agents lost their observations and that at least one agent still receives a true observation at every step, because the correction block is only defined when at least one real observation exists.","fun_headline_variants_meta":{"raw":{"variants":["RMIO keeps MARL agents steady when observations drop out","Model-based MARL fills missing observations for robust teamwork","MARL with lost observations? RMIO rebuilds them from others","RMIO: world model reconstructs lost agent views on the fly","For MARL, RMIO turns missing observations into no problem"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000631,"raw_usage":{"total_tokens":2915,"prompt_tokens":947,"completion_tokens":1968,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":1897}},"tokens_in":563,"tokens_out":1968,"duration_ms":11488,"temperature":1.0,"reasoning_tokens":1897,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:59:14.122596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would run RMIO in the same benchmarks with a protocol in which all agents lose observations simultaneously at some steps, so $m=0$; the correction block's loss is undefined for $m=n$, and the paper neither defines this case nor gives an implementation of the detector that decides which agents lost observations. If the announced robustness at $p_{\\text{loss}}=1$ relies on the random-subset protocol always leaving one observer and a perfect detector, those runs should degrade sharply.","supporting_citations":[{"cited_title":"Samvelyan, T","cited_arxiv_id":null,"evidence_quote":"Provides the SMAC benchmark and its maps used for the discrete cooperative-control experiments."},{"cited_title":"Egorov, A","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-agent recurrent world model and the CTCE baseline that RMIO adapts and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides a model-based MARL baseline that treats local models as decision-making agents and is used in standard and observation-loss comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the policy-gradient algorithm that RMIO adopts as its actor-critic policy model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MaMuJoCo benchmark and one of the compared model-free baselines for continuous-control experiments."},{"cited_title":"Hafner, T","cited_arxiv_id":null,"evidence_quote":"Supplies the discrete latent world-model design that the multi-agent predecessor is built on and that RMIO adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the reward-smoothing approach that RMIO adopts to make reward modeling more stable."}],"review_version":1}