{"id":"691fccda-fa5e-46b1-9a86-85bc592110f0","arxiv_id":"2501.15695","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A Dec-MARL framework with goal-aware and time-aware knowledge sharing is proposed, but its reported gains are contradicted by the paper's own tables.","lead":"This paper proposes a decentralized multi-agent reinforcement learning framework that combines communication and coordination with goal-aware and time-aware knowledge sharing. The authors report performance gains, but their own experimental tables show the full method is rarely better than an independent baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The final model A5 is worse than the independent A1 baseline in all four Large-environment cells, so the central claim of significant enhancement is contradicted by the paper's own tables.","rationale":"I read the paper's central claim as an empirical one: A5 (goal- and time-aware communication and coordination) should outperform independent agents and ablations. The most load-bearing test is therefore not the theoretical form of Equation 3 but whether the reported experiments show the claimed effect. I checked Tables I and II cell by cell. Contrary to the reader's summary, A5 beats A1 in four of eight printed cells, not one; however, the four losses are exactly the four Large-environment cells, including both dynamic-obstacle (Hard) settings. This sign reversal is a direct contradiction of the abstract's 'significantly enhances overall performance.' Since the paper provides no statistical tests, no seed count, and no code, the mixed table cannot be rescued by an appeal to significance. The reader's weakest_assumption about the time-aware intrinsic reward is plausible but secondary: even a perfectly specified intrinsic reward would not establish the central claim given the Large-environment results. I therefore concur with REJECT, with the caveat that the strongest justification is the internal empirical contradiction rather than the mathematical form of Equation 3. A rerun with seeds and paired intervals would settle the issue.","tokens_in":12534,"tokens_out":7918,"duration_ms":75441,"concrete_test":"Obtain the authors' per-seed per-episode rewards (or rerun their implementation, since no code is provided) for A5 and A1 in all eight environment-scenario cells, using at least 30 independent seeds. Compute paired differences A5 minus A1 with 95% bootstrap confidence intervals for each cell. If the Large-Easy and Large-Hard cells yield confidence intervals that exclude a positive effect (as the printed means strongly suggest), the claim that the full framework significantly enhances overall performance is falsified. If the authors cannot supply per-seed data or code, the central claim remains unverifiable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the complete framework (A5) significantly enhances performance through goal-aware and time-aware knowledge sharing. The printed results contradict this. In every Large-environment cell, A5 is worse than the plain independent baseline A1: Table I Large-Easy 0.229 vs 0.233, Large-Hard 0.235 vs 0.243; Table II Large-Easy 0.232 vs 0.249, Large-Hard 0.233 vs 0.251. Across all eight cells, A5 surpasses A1 only in the four Base environments, and A5 is never best in a Large environment. Moreover, in Scenario 1 Base-Hard, the simpler A2 (0.207) beats A5 (0.191). A5 is the best-performing agent type in only two of eight cells. Thus the framework's benefit, if any, reverses with environment scale, and the dynamic-obstacle settings where coordination is supposed to matter most are exactly where the full model underperforms. The Section V-C.3 conclusion that 'significant improvements' are demonstrated is unsupported by the tabulated means; no statistical test, number of seeds, or code is provided to transform these mixed signs into a claim of significance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Decentralized Multi-Agent Reinforcement Learning (Dec-MARL) framework that integrates peer-to-peer communication and coordination with goal awareness and time awareness. The method introduces a mental-state representation, a time-aware intrinsic reward, and a three-phase Share-Reason-Aggregate mechanism for knowledge sharing. The authors evaluate five agent variants (A1 through A5) in 10x10 and 20x20 grid-world environments with static or dynamically appearing obstacles, under two goal scenarios. The central claim is that the full framework, A5, significantly improves overall performance over independent agents.","tokens_in":12919,"tokens_out":4036,"duration_ms":35399,"significance":"If the claimed improvements were real, the framework would be a useful contribution to decentralized MARL with heterogeneous goals and dynamic obstacles. The paper's modular ablation design from A1 to A5 and its explicit treatment of information decay are appropriate and address an important problem. However, the reported evidence does not establish the central claim: the full model A5 is worse than the independent baseline A1 in all four Large-environment cells, no statistical testing is provided, and the main hyperparameters are tuned on the same tasks used for evaluation. The contribution is therefore not currently substantiated.","major_comments":[{"comment":"The central claim that A5 significantly enhances performance is contradicted by the reported means. In Table I, A5 is worse than A1 in Large-Easy (0.229 vs 0.233) and Large-Hard (0.235 vs 0.243); in Table II, A5 is worse than A1 in Large-Easy (0.232 vs 0.249) and Large-Hard (0.233 vs 0.251). Across the eight settings, A5 is best only in Base-Easy (0.139) and Scenario 2 Base-Hard (0.225), and in Scenario 1 Base-Hard a simpler variant A2 (0.207) outperforms A5 (0.191). The tables report only Roverall, yet Section V-C.1 claims '5% fewer steps' and '15% fewer steps' without any step-based results, and no number of seeds or statistical tests are given. These results do not support the abstract's assertion of significant enhancement.","section":"Section V-C, Tables I and II"},{"comment":"Several core hyperparameters are chosen by 'empirical analysis' on the same environments used for evaluation: alpha=0.1 in Eq. (5), the time increment 0.01 for dt in Eq. (3), the Jaccard threshold J<=0.5 in Eq. (7), and beta=0.1 in Eqs. (8)-(9). Because the reported A5 gains depend on these values and no held-out validation or sensitivity analysis is provided, the improvements are fitted to the test tasks rather than independent predictions. This circularity is load-bearing for the claim of general enhancement.","section":"Section IV-B and Section V-B"},{"comment":"Equation (3) is ambiguous and unverified. The expression for u_t_i(o) is not typeset unambiguously (it is unclear whether the exponent is 1/(2 dt') or dt'/2), and the stated condition that the value of an observation decays over time is not demonstrated against the definition of dt'. Since the time-aware intrinsic reward is the key mechanism intended to improve exploration in dynamic environments, its behavior must be specified precisely and validated with a clear derivation or experiment.","section":"Section IV-B, Eq. (3)"},{"comment":"The parameter-averaging update is acknowledged in the text to 'sometimes lead to situations where poor-performing agents negatively impact the performance of others,' and beta is merely set low. No mechanism or experiment isolates when averaging helps versus hurts, so the ablation from A3 to A4 to A5 cannot separate the negative effect of aggregation from the benefit of communication. This weakens the interpretation of the A4 and A5 results.","section":"Section IV-C.3, Eqs. (8)-(9)"}],"minor_comments":[{"comment":"There is a typo 'Muti-Agent' in the Conclusion; 'relay buffer' in Section V-B should be 'replay buffer'; and Section V-B contains an incomplete sentence, 'we conducted our experiments a complex 2D environment'.","section":"Abstract/Conclusion"},{"comment":"The symbol S is overloaded: it denotes the set of states in Section III and the aggregation function in Eq. (1), which is confusing for the reader.","section":"Section III and Section IV-A"},{"comment":"The axes and the relationship between dt' and time steps are not labeled in Figure 3, making the illustrative '100 steps' difficult to interpret.","section":"Figure 3"},{"comment":"The text reports percentage reductions in steps, but the tables report only Roverall; a separate table of step counts or episode-completion statistics is needed to support those claims.","section":"Section V-C.1"},{"comment":"No code or data availability statement is included, which hampers reproducibility of the experimental results.","section":"Overall"}],"recommendation":"reject","confidential_remarks":"The empirical core of the paper is not supported by its own tables: the full model A5 is worse than the independent baseline A1 in all four Large environments, which are exactly the settings where coordination and time-awareness are claimed to matter most. The hyperparameter tuning on the evaluation tasks further compounds the problem. I would only consider a revised version if new experiments show A5 beating A1 in Large environments with proper seed variance, an evaluation protocol that separates hyperparameter selection from testing, and a corrected presentation of the intrinsic reward equation. The current version is not a candidate for a minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of arXiv:2501.15695. The framework is a genuine combination: UVFA-style goal-conditioned policies, a time-decay intrinsic reward, goal-aware peer/adviser selection under Jaccard overlap, and parameter averaging, all in a fully decentralized DTDE setting. I haven't seen this exact package in the literature, and the ablation ladder A1–A5 is a sensible way to test each component. The paper also does something I respect: it explicitly admits in Section IV-C.3 that parameter averaging can let poor performers drag others down, and it recommends keeping beta low.\n\nBut the central claim doesn't survive contact with the paper's own tables. The abstract says goal-aware and time-aware knowledge sharing 'significantly enhances overall performance.' In Scenario 1, A5 beats the independent A1 baseline in Base-Easy and Base-Hard, but loses in Large-Easy (0.229 vs 0.233) and Large-Hard (0.235 vs 0.243). In Scenario 2, same pattern: A5 wins in Base-Easy and Base-Hard, loses in Large-Easy (0.232 vs 0.249) and Large-Hard (0.233 vs 0.251). So across all eight environment/agent cells, A5 is best only in two, and it is never best in a Large environment. The text's claim that 'A4 and A5 excel in Large-Easy and Large-Hard' is the opposite of what Table I shows—in Large-Easy A3 is best, in Large-Hard A1 is best. That's a direct misreading of the results, and it undermines the 'significant improvement' conclusion.\n\nThere are other problems. The hyperparameters alpha, beta, J-threshold, and dt increment were chosen by 'empirical analysis' on the same test environments, so the reported gains are fitted, not predicted. No number of seeds, no statistical tests, and no code are provided, so we can't tell whether the mixed signs are noise. Equation 3 for the novelty utility is ambiguous as printed: the exponent is unclear. And the parameter averaging in Equations 8 and 9 is a known-fragile operation, which the authors themselves concede.\n\nWhat's worth keeping: the problem is real, the framework design is thoughtful, and the ablation structure is the right way to ask the question. But as it stands, the evidence base does not support the abstract's claim. A revision would need a proper experimental protocol—multiple seeds, significance tests, held-out hyperparameters, and at least one external baseline beyond DDPG variants—before the empirical story could be trusted.\n\nFor a reading group, it's a decent example of why you should check tables against prose. I wouldn't cite it in its current form, and I'd advise a serious editor to desk-reject rather than spend referee time on a paper whose own results contradict its headline.","headline":"A thoughtful framework, but the paper's own tables contradict its claim of significant improvement—A5 beats independent A1 only in small environments.","tokens_in":13361,"tokens_out":3882,"would_cite":false,"duration_ms":32363,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In decentralized multi-agent reinforcement learning, sharing knowledge filtered by goal relevance and freshness significantly improves performance in dynamic environments.","keywords":["multi-agent reinforcement learning","decentralized training and decentralized execution","goal awareness","time awareness","intrinsic reward","communication and coordination","knowledge sharing","dynamic obstacles"],"falsifier":"In the Easy environments (static obstacles), remove the time-aware intrinsic reward entirely; if the full A5 framework still shows the same margin over the A4 ablation, then the paper's central attribution of gains to time awareness is not supported.","tokens_in":12362,"feed_emoji":"🤖","tokens_out":8141,"duration_ms":65553,"temperature":0.7,"pith_summary":"The paper proposes a framework for decentralized multi-agent reinforcement learning where each agent pursues its own goal with limited visibility. The central claim is that agents learn better when they filter shared knowledge by two criteria: whether the information is relevant to the receiver's goal and how recently the information was obtained. The authors build this into a Share-Reason-Aggregate cycle, where peers and advisors exchange observations, an agent reasons about stale versus fresh knowledge, and selected peers average their learning parameters. They test the framework on grid-world tasks with dynamically appearing obstacles and report that the full system outperforms independent agents and most ablations. If the claim holds, it suggests that context-aware communication can replace centralized coordination in decentralized teams.","feed_headline":"Time-aware, goal-aware agents outperform independent learners","feed_subtitle":"Agents that weigh goal relevance and information freshness outperform independent learners in dynamic obstacle grids.","key_machinery":"The machinery is the Share-Reason-Aggregate cycle with a time-aware mental state. An agent's mental state $M_i$ stores masked labels and visit durations; the novelty score $u_t^i(o) = e^{\\frac{1}{2}d_{t'}}$ from Equation 3 makes old observations decay and rediscovered ones fresh again. This score enters the mental-state embedding (Equation 4) and an intrinsic reward (Equation 5) that balances extrinsic goal reward against exploration. In a session the agent broadcasts its goal to identify peers and advisors, merges shared observations by freshness (Equation 6), selects peers by Jaccard overlap (Equation 7), and averages learning parameters (Equations 8 and 9) with a low blending factor $\\beta$.","core_discovery":"The authors claim that a fully decentralized MARL framework combining goal-aware communication with time-aware coordination improves both exploration and task success compared with independent agents and partial ablations. The strongest form of the claim is carried by the full A5 agent, which broadcasts its goal to separate same-goal peers from advisors, merges only fresh observations into its mental state, and averages parameters only with peers whose mental-state overlap is low. In the paper's experiments, A5 is the best or near-best performer in most tested environments and completes tasks in fewer steps than the independent baseline, with the largest gains in hard, dynamic environments. The work is presented as a demonstration that context—what the receiver wants and how old the information is—should determine what is shared, not just whether to communicate.","pith_inferences":["One extension not tested by the paper: replacing the hand-set time-decay formula with a learned or count-based staleness measure, to see whether the exponential form is essential to the reported gains.","The Jaccard-overlap rule for peer selection could be replaced by an estimate of each peer's actual contribution; the paper's own admission that poor agents hurt others suggests adaptive weighting would stabilize aggregation.","Because agents share an ontology and a bounded map, goal-aware filtering might transfer to semantic labels or learned representations, but only if the mental-state encoding is learned rather than predefined."],"forward_implications":["A fully decentralized team can match or beat independent learners without a centralized critic, shared reward, or hand-designed communication protocol.","Goal-aware filtering matters in mixed-goal scenarios: agents that always share regardless of goal (A4) sometimes perform worse than independent agents, while goal-aware A5 is more consistent.","Time awareness is most valuable where knowledge goes stale: in large or hard environments A5 beats A4, while in small static environments mental state alone can suffice.","Advisors contribute heuristic shortest-path plans rather than learned parameters, so advice can help without transferring goal-specific policies.","The parameter-averaging step should be used with low $\\beta$, since the paper observes that poor-performing peers can negatively affect others."],"supporting_citations":[{"why":"Supplies the actor-critic baseline (MADDPG) that the A1 independent agent instantiates in a fully decentralized way.","marker":"[8]"},{"why":"Provides the count-based novelty-sharing intrinsic reward that the time-aware Equation 3 is designed to replace.","marker":"[35]"},{"why":"Supplies the heuristic shortest-path planning used by advisors during the coordination phase.","marker":"[39]"},{"why":"Provides the universal value function approximation formulation used to condition policies on individual goals.","marker":"[43]"},{"why":"Supplies the DDPG implementation details (network sizes, target updates, optimizer) used by every agent.","marker":"[45]"}],"fun_headline_variants":["Context-aware knowledge sharing beats independent agents","Goal and time awareness improve multi-agent coordination","Decentralized agents excel with contextual knowledge sharing","Fresh, relevant data boosts collaborative task success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on two assumptions: a simple time-decay formula can correctly measure how stale an agent's knowledge is, and averaging learning parameters with peers chosen by overlap helps more than it hurts.","fun_headline_variants_meta":{"raw":{"variants":["Context-aware knowledge sharing beats independent agents","Goal and time awareness improve multi-agent coordination","Decentralized agents excel with contextual knowledge sharing","Fresh, relevant data boosts collaborative task success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1185,"prompt_tokens":878,"completion_tokens":307,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":252}},"tokens_in":494,"tokens_out":307,"duration_ms":3648,"temperature":1.0,"reasoning_tokens":252,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:02:19.003826+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the Easy environments (static obstacles), remove the time-aware intrinsic reward entirely; if the full A5 framework still shows the same margin over the A4 ablation, then the paper's central attribution of gains to time awareness is not supported.","supporting_citations":[{"cited_title":"Settling decentralized multi-agent coordinated exploration by novelty sharing,","cited_arxiv_id":null,"evidence_quote":"Provides the count-based novelty-sharing intrinsic reward that the time-aware Equation 3 is designed to replace."},{"cited_title":"Learn to follow: Decentralized lifelong multi-agent pathfinding via planning and learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the heuristic shortest-path planning used by advisors during the coordination phase."},{"cited_title":"Universal value func- tion approximators,","cited_arxiv_id":null,"evidence_quote":"Provides the universal value function approximation formulation used to condition policies on individual goals."},{"cited_title":"Continuous control with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the DDPG implementation details (network sizes, target updates, optimizer) used by every agent."}],"review_version":1}