{"id":"7c0747e9-151d-476b-b4d6-af523fec1130","arxiv_id":"2412.15639","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SICA combines selective state-space filtering with attention-based training-time communication and a regeneration module to let MARL agents coordinate without messages at execution time.","lead":"This paper introduces SICA, a training scheme for cooperative AI agents that lets each agent privately filter useful information and gradually learn to guess teammates' intentions, so no communication is needed at deployment. It reports higher win rates than several standard algorithms on StarCraft, SMACv2, and football benchmarks, but one added experiment shows the plug-in can hurt performance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regeneration of teammates' hidden states from a single local history is the load-bearing step (Eqs.","rationale":"The central constructive claim is that a decentralized policy can regenerate enough of the centralized attention information to coordinate without communication. The paper's formal mechanism for this is Eq. 12 with the alpha schedule in Eqs. 9-10. That mechanism is not a proven property; it is an empirical identifiability and transfer assumption. The paper's own Section 6 admits the failure mode, and the reported t-SNE visualization is not a quantitative check. I therefore agree with the reader's weakest_assumption. I would not change the CONDITIONAL verdict: the SMAC, SMACv2, and GRF results are consistent with the mechanism working in those particular settings, but the absence of code, hyperparameters, and a distribution-shift sensitivity study means the general plug-and-play claim is not yet established. The Table 2 discrepancy, where SICA-MADDPG is numerically worse than MADDPG in all three reported tasks, is a separate internal-consistency issue that further supports caution but does not by itself refute the benchmark-specific results.","tokens_in":11876,"tokens_out":6687,"duration_ms":58548,"concrete_test":"On SMACv2 protoss_5_vs_5, train SICA and every 100k steps freeze the policy, then evaluate on held-out episodes under three execution modes: (i) alpha=1 (use true v_i), (ii) alpha=0 (use regenerated v_hat_i), and (iii) alpha=0 but with the true v_i provided (oracle). Also record the per-agent Eq. 12 error on these held-out rollouts as a function of training time. If (i) or (iii) significantly beats (ii), or if the alignment error grows during training, then the tacit-learning claim fails under distribution shift; if (ii) matches (i) and the error stays flat, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"At inference, alpha(t)=0 (Eq. 10), so every agent's Q-value is computed from the regenerated vector v_hat_i, a reconstruction from its own observation-action history. But the target v_i in Eq. 12 is the attention-weighted sum of all other agents' hidden states h_j (Eqs. 7-8), hence a function of the joint hidden state. In a Dec-POMDP with stochastic unit compositions (SMACv2) or heterogeneous roles, the local history does not generally identify the joint hidden state: multiple joint states consistent with tau_i produce different v_i, making the regression target in Eq. 12 ill-posed. The alignment loss minimizes only an expected squared error and cannot enforce pointwise accuracy under the policy shift induced by the alpha schedule. Section 6 explicitly concedes that the Regeneration Block 'may struggle when agents' observations or trajectories are dissimilar.' The only evidence offered, the t-SNE visualization in Figure 10, is qualitative; no numerical regeneration error, no per-agent breakdown, and no sensitivity of win rate to alpha are reported. Without code or hyperparameters, this transfer assumption is the least secured part of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SICA, a modification of the agent networks used in CTDE value-decomposition MARL methods such as QMIX and VDN. SICA adds three components: a Selection Block (an S6 selective state-space layer with a gating unit and a mini-buffer of recent observation-action pairs), a Communication Block (attention over agents' hidden states to produce a 'true information' vector during training), and a Regeneration Block (an MLP that reconstructs the true information from a single agent's local history). A cosine-annealed weight alpha interpolates between the true information and the regenerated estimate during training, reaching zero at inference so that execution is communication-free. The loss is the sum of the standard TD loss and an alignment loss whose weight follows a threshold schedule. Experiments on SMAC, SMACv2, and GRF are reported against VDN, QMIX, QTRAN, NDQ, QMIX-Attention, CDS-QMIX, and QMIX-CADP, along with ablations for the selection block and for the progressive alpha schedule. The paper also shows an application to VDN and to MADDPG. Section 6 lists limitations, including the fragility of the regeneration block when agents' observations or trajectories are dissimilar and the need to tune several hyperparameters.","tokens_in":12147,"tokens_out":5054,"duration_ms":42473,"significance":"If the core result holds, SICA provides a generic upgrade for CTDE agent networks that removes execution-time communication while retaining train-time coordination, and it adds an information-selection mechanism that is currently absent from standard CTDE methods. The paper deserves credit for evaluating against a broad set of baselines, including explicit communication methods, and for running ablations that isolate the contribution of the selection block and of progressive information regeneration. Those ablations are a genuine asset. However, the significance is materially weakened by the contradiction in Table 2, the qualitative-only evidence for the central regeneration mechanism, and the absence of code and hyperparameter details. The three-seed confidence intervals are too thin to support the strongly worded 'consistently outperforms' claims. The central value-decomposition comparisons for QMIX and VDN are internally coherent, but the broader plug-and-play generality claim is not established.","major_comments":[{"comment":"The claim that 'SICA continues to enhance the overall performance of the framework' is directly contradicted by the table: SICA-MADDPG has lower mean return than MADDPG in all three environments (Predator-Prey 725.11 vs 831.48, Navigation -255.23 vs -233.31, Pantomime -926.55 vs -812.79) and substantially larger standard deviations. This undermines the plug-and-play generality claim in the Abstract and Section 1. The authors should either remove the MADDPG claim, restrict the plug-and-play claim to value-decomposition methods, or explain why the actor-critic setting is outside the intended scope.","section":"Section 5.3, Table 2"},{"comment":"The regeneration mechanism is the load-bearing step for decentralized execution, but it is validated only qualitatively. The target v_i in Eq. 12 is the attention-weighted sum of all other agents' hidden states (Eqs. 7-8), while the regressor \\hat v_i uses only agent i's local observation-action history. In stochastic environments such as SMACv2, multiple joint hidden states can be consistent with the same local history, making the regression target ill-posed, and the alignment loss minimizes only an expected squared error, not pointwise accuracy. Since alpha(t)=0 at inference (Eq. 10), the entire decentralized Q-value depends on this reconstruction. Figure 10 shows a t-SNE visualization but reports no numerical regeneration error, no per-agent breakdown, and no sensitivity to the alpha schedule or alignment weights. The paper needs quantitative evidence that the reconstruction remains accurate under the policy shift induced by the alpha schedule, or it should temper the claim that SICA 'relies solely on local information' without loss of coordination quality.","section":"Section 4.3, Eq. 12; Section 6"},{"comment":"The manuscript does not provide code or hyperparameter values. The free parameters include the mini-buffer capacity b, the alignment loss weights beta_1 and beta_2 and threshold T in Eq. 14, the alpha schedule endpoints in Eq. 10, network dimensions, learning rates, and the unspecified schedule by which the Regeneration Block's mini-buffer is reduced from other agents' observation-action pairs to zero (Section 4.2). Without these details, the experiments cannot be reproduced or independently checked, and the claim of 'seamless integration' with any value decomposition method is not verifiable. At minimum, a supplementary table of hyperparameters and a statement about which SMAC/SMACv2 version and reward settings were used are necessary.","section":"Reproducibility"},{"comment":"All learning curves are reported as means over three seeds with 95% confidence intervals. With three seeds, the confidence interval is highly sensitive to a single outlier and does not support the language 'SICA consistently outperforms the baselines across all maps.' The paper should report the number of runs per cell, use a more robust uncertainty estimate (e.g., interquartile range or individual runs overlaid), or apply a paired statistical test across maps. This issue is directly relevant to the headline comparison claims in Figures 3 and 4.","section":"Section 5, statistical support"}],"minor_comments":[{"comment":"There is a typo 'clasSICAl' in the enumeration of baselines; 'classical' is intended.","section":"Section 5.1"},{"comment":"The word 'decompostition' appears in the sentence 'can be seamlessly integrated with any value decompostition method'; this should be 'decomposition'.","section":"Section 4"},{"comment":"The citation for Google Research Football is given as [9], but reference [9] is 'It's Raw! Audio Generation with State-Space Models'; the GRF paper by Kurach et al. is reference [18]. Also, references [40] and [41] are the same NDQ paper (Learning Nearly Decomposable Value Functions Via Communication Minimization) and should be merged.","section":"References"},{"comment":"The notation Split(x_t^i) is not defined, and the intermediate variable z_t^i is introduced without explanation; please clarify what the split is and how the two MLP outputs interact.","section":"Eq. 6"},{"comment":"The mini-buffer is described as containing the preceding b observation-action pairs, but the listed set is {(o_{t-1}^i, u_{t-2}^i), ..., (o_{t-b}^i, u_{t-b-1}^i)}, which pairs each observation with the action from the previous time step; clarify whether this temporal offset is intentional or a typo.","section":"Section 4.2"},{"comment":"The caption uses 'the Nth agent' where 'agent i' is meant; the figure is also quite small and the connection between the Regeneration Block and the agent value head is hard to follow.","section":"Figure 2"},{"comment":"The abstract states that agents rely 'solely on local information' without adding 'during execution'; since training uses global information through the Communication Block, the phrasing should be tightened to avoid ambiguity.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper's SMAC/SMACv2/GRF comparisons for QMIX and VDN are internally coherent, and the ablations for the selection block and the alpha schedule are useful. The principal problems are the Table 2 contradiction with the plug-and-play claim, the insufficient validation of the regeneration mechanism, and the lack of code/hyperparameters. The regeneration issue is the one I would most want the authors to address with quantitative evidence, because it is the linchpin of the decentralized execution claim. I do not see evidence of citation manipulation; the duplicate NDQ reference is an editing error. The paper fits AAMAS's scope, but it currently overstates its generality relative to what is demonstrated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things worth knowing about this paper. First, the mechanism is well-motivated: train with attention-based communication, then anneal in a local \"regeneration\" of the true information so execution becomes communication-free. The ablations (SICA-ZERO, SICA-ONE) actually support that the progressive schedule matters, not just the architecture. Second, the paper overclaims. Table 2 shows SICA-MADDPG worse than MADDPG on all three tasks, while the text says it \"continues to enhance overall performance.\" That is a direct contradiction, and it undermines the \"seamlessly integrated with any value decomposition method\" selling point.\n\nWhat is genuinely new: the S6 selection block combined with alpha-annealed regeneration of attention-weighted teammate hidden states is not in COLA, TACO, or QMIX-CADP. The SMAC/SMACv2/GRF results are internally coherent, and SICA beats QMIX and VDN baselines across maps. The paper also has an honest limitations section, which is more than many MARL submissions offer.\n\nThe soft spots are real, but not evenly distributed. The Table 2 issue is a clear error the authors should have caught. No code and no hyperparameters make the results impossible to check. Three seeds is thin but typical for this field. The bigger concern is the load-bearing regeneration step: at inference each agent uses only its local history to reconstruct a global attention target. The stress-test worry that this target is ill-posed when histories do not identify teammates' hidden states is legitimate. The paper's own limitation section concedes the regeneration block may struggle when trajectories are dissimilar. The only evidence is a qualitative t-SNE visualization; no numerical regeneration error, no per-agent breakdown, no sensitivity to alpha. That is the least secured part of the central claim.\n\nStill, the central comparison for QMIX/VDN on standard benchmarks is defensible, and the explicit-to-tacit schedule is a sensible idea that the ablation studies support. This paper is for MARL researchers working on communication-free execution or value decomposition. It deserves a serious referee, but the authors need to fix the Table 2 contradiction, release code and hyperparameters, and add quantitative regeneration validation before it is solid.\n\nMy call: send it to review rather than desk reject, but expect substantial revision. I would not cite it in the next 12 months until the code and the regeneration evidence are actually available.","headline":"A plausible explicit-to-tacit value-decomposition extension whose core regeneration assumption is under-supported, and whose own Table 2 contradicts the plug-and-play claim.","tokens_in":12600,"tokens_out":2007,"would_cite":false,"duration_ms":17504,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that cooperative MARL agents can learn implicit coordination without communication by regenerating a teammate-information summary from local histories, and that this improves CTDE baselines and beats explicit…","keywords":["multi-agent reinforcement learning","tacit learning","adaptive information selection","centralized training decentralized execution","implicit communication","value decomposition","selective state space model","StarCraft Multi-Agent Challenge"],"falsifier":"Train SICA on a task where each agent's observations are deliberately stripped of any information about teammates, such as agents in separate rooms sharing only a team reward; if the win rate collapses when the training-time communication is annealed away but stays high when the true information is available, then the regeneration assumption fails.","tokens_in":11662,"feed_emoji":"🤝","tokens_out":11462,"duration_ms":88010,"temperature":0.7,"pith_summary":"The paper proposes SICA, an add-on architecture for cooperative multi-agent reinforcement learning under the centralized-training/decentralized-execution paradigm. SICA's goal is to let each agent select the most decision-relevant parts of its observation history and to learn a tacit understanding of teammates: during training, an attention mechanism shares hidden states among agents, while a regeneration block learns to reproduce that shared information from a single agent's local history alone. The shared information is blended with the local reconstruction through a cosine-annealed schedule, so at execution time no communication is needed. On SMAC, SMACv2, and Google Research Football, the authors report that SICA improves value-decomposition baselines such as QMIX and VDN and matches or surpasses explicit communication methods.","feed_headline":"Silent teamwork beats chatty agents in cooperative AI benchmarks","feed_subtitle":"SICA filters observations and rebuilds teammate info locally, beating explicit communication on SMAC, SMACv2, and GRF.","key_machinery":"The load-bearing mechanism is the train-time/execution-time division of labor around the true information $v_i = \\sum_{i\\neq j} w_{i,j} h_j$, an attention-weighted combination of other agents' hidden states. During training, the Communication Block provides $v_i$; the Regeneration Block must learn to produce $\\hat{v}_i$ from local observation-action history, and the cross-information $\\bar{v}_i = (1-\\alpha(t))\\hat{v}_i + \\alpha(t) v_i$ feeds the Q-value. The Selection Block uses a gated MLP plus an S6 selective state-space layer with input-dependent $\\Delta$, $B$, and $C$ parameters to filter irrelevant information from a mini-buffer of past observation-action pairs. The cosine annealing of $\\alpha(t)$ and the thresholded weighting $\\sigma(t)$ of the alignment loss are what make the transition from centralized to decentralized execution smooth rather than abrupt.","core_discovery":"The central claim is that the two bottlenecks of CTDE, agents' inability to filter input information and their inability to access global information during execution, can be overcome jointly. SICA adds three blocks to each agent network: a Selection Block (two MLPs and an S6 selective state-space layer) that gates the observation-action history and decides what to remember; a Communication Block that computes an attention-weighted sum of other agents' hidden states to form the true information $v_i$; and a Regeneration Block that maps the agent's own history to an estimate $\\hat{v}_i$. The alignment loss $\\mathcal{L}_{Align}$ drives $\\hat{v}_i$ toward $v_i$, while the cross-information blend uses a coefficient $\\alpha(t)$ annealed from 1 to 0, so the architecture gradually stops relying on the communicated target. The paper reports that this yields better win rates than QMIX, VDN, QTRAN, QMIX-Attention, and NDQ on hard and super-hard SMAC maps, higher scores than CDS-QMIX and QMIX-Attention on GRF, and consistent superiority in SMACv2; ablations show both the Selection Block and progressive regeneration contribute.","pith_inferences":["A testable implication not explored in the paper is that the alignment loss acts as an information bottleneck: the local history must retain exactly the features of teammates' behavior that the attention target encodes, so measuring the mutual information between $\\hat{v}_i$ and $v_i$ along training could reveal when the reconstruction becomes the limiting factor.","Because the regeneration target $v_i$ is itself learned through attention weights that shift during training, the method's success may depend on target stability; comparing cosine annealing against a constant small $\\alpha$ in tasks with non-stationary teammates would isolate that dependence.","The paper evaluates discrete-action benchmarks; a natural next test is continuous-action domains, where the regenerated target would need to represent teammate policies rather than action-value information."],"forward_implications":["SICA can be dropped into value-decomposition learners such as QMIX and VDN, or actor-critic learners such as MADDPG, and improve them without changing the base algorithm's training objective beyond the added alignment loss.","Execution becomes communication-free, so the method avoids message latency and applies where explicit channels are unavailable or too costly.","On SMACv2 with larger team sizes, SICA outperforms the explicit-communication baseline QMIX-Attention, supporting the claim that the selection mechanism helps as the volume of information grows.","The gradual annealing from true to regenerated information outperforms both no regeneration (SICA-ZERO) and late switching (SICA-ONE), so the smooth transition is load-bearing for the method."],"supporting_citations":[{"why":"Supplies the QMIX value-decomposition framework and TD loss that SICA wraps, including the monotonic mixing network.","marker":"[28]"},{"why":"Provides the S6 selective state-space layer used in the Selection Block and Regeneration Block to filter observation-action history.","marker":"[10]"},{"why":"Defines the VDN value-decomposition baseline that SICA is shown to improve when applied as SICA-VDN.","marker":"[32]"},{"why":"Defines the SMAC benchmark, whose hard and super-hard maps provide the main win-rate comparisons against baselines.","marker":"[29]"},{"why":"Defines SMACv2, the stochastic benchmark used for the main results and the ablation studies.","marker":"[6]"},{"why":"Defines the Google Research Football environment used for comparisons against CDS-QMIX and QMIX-Attention.","marker":"[18]"},{"why":"Introduces the explicit-to-tacit cooperation paradigm that SICA extends with selection and regeneration.","marker":"[20]"},{"why":"Provides the QMIX-CADP communication-free execution baseline, used to justify gradual rather than direct pruning.","marker":"[45]"},{"why":"Provides the QMIX-Attention explicit communication baseline used across SMAC, SMACv2, and GRF comparisons.","marker":"[14]"},{"why":"Supplies the scaled dot-product attention formula used in the Communication Block to compute weights $w_{i,j}$ and true information $v_i$.","marker":"[36]"}],"fun_headline_variants":["Silent coordination wins in multi-agent benchmarks","Adaptive info filtering boosts cooperative AI","SICA: tacit learning beats chatty MARL agents","No chatter, better teamwork: new MARL framework","Selective observation sharpens multi-agent decisions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that carries the method is that a single agent's own local observation-action history contains enough information about its teammates' behavior to reconstruct the shared message, and that this stays true while the agents' policies change during training.","fun_headline_variants_meta":{"raw":{"variants":["Silent coordination wins in multi-agent benchmarks","Adaptive info filtering boosts cooperative AI","SICA: tacit learning beats chatty MARL agents","No chatter, better teamwork: new MARL framework","Selective observation sharpens multi-agent decisions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1259,"prompt_tokens":976,"completion_tokens":283,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":213}},"tokens_in":592,"tokens_out":283,"duration_ms":3503,"temperature":1.0,"reasoning_tokens":213,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:14:07.204750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SICA on a task where each agent's observations are deliberately stripped of any information about teammates, such as agents in separate rooms sharing only a team reward; if the win rate collapses when the training-time communication is annealed away but stays high when the true information is available, then the regeneration assumption fails.","supporting_citations":[{"cited_title":"From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARL","cited_arxiv_id":"2304.14656","evidence_quote":"Introduces the explicit-to-tacit cooperation paradigm that SICA extends with selection and regeneration."},{"cited_title":"In International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Provides the QMIX-CADP communication-free execution baseline, used to justify gradual rather than direct pruning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the QMIX-Attention explicit communication baseline used across SMAC, SMACv2, and GRF comparisons."}],"review_version":1}