{"id":"13b73357-3d85-418b-a22c-af89df223762","arxiv_id":"2501.02174","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"TACTIC uses offline contrastive pretraining, aligning integrated local observations and messages with each agent's egocentric state, to improve multi-agent coordination across varied sight ranges on SMACv2.","lead":"TACTIC is a communication method for cooperative multi-agent reinforcement learning that is pretrained offline with contrastive learning, using the global state as a training target, then frozen during policy learning. It is designed to keep agents coordinated when their visibility range at deployment differs from the range seen in training, and tests on SMACv2 show better generalization than QMIX-based baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The random sight-range r used in offline pretraining (Sec. 4.1) is never specified or ablated; if it covers the [0.2,5] evaluation grid, the reported 'generalization' is interpolation via data coverage, not extrapolation, and the baselines lack this augmentation.","rationale":"The reader's weakest assumption identifies essentially the same concern: the distribution over r is unspecified, so the claimed cross-sight-range generalization may be interpolation within a seen distribution. I agree this is the most load-bearing issue. The paper's empirical results are plausible and the contrastive framework is coherent, but the absence of any specification or ablation of the random-r augmentation leaves the central interpretation ambiguous. The correct response is to keep the conditional verdict: the authors should disclose the r distribution and run the held-out/r-fixed pretraining experiments before the generalization claim can be accepted at face value. I do not see an internal inconsistency that would justify rejection, and the reader's conditional framing already accounts for the main technical risk.","tokens_in":22444,"tokens_out":5467,"duration_ms":60904,"concrete_test":"Rerun TACTIC's offline pretraining with r sampled only from a narrow interval around each online training SRR, e.g., r in [0.18, 0.22] for the SRR=0.2 policy, and evaluate on the full [0.2,5] grid. If the heatmaps collapse to baseline-like diagonal behavior, the reported generalization depends on pretraining having already seen the test sight ranges. As a complementary check, train QMIX with the same random-sight-range augmentation over [0.2,5]; if it matches TACTIC's off-diagonal performance, the advantage is augmentation rather than task-agnostic communication.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 describes Feature Integration Alignment (FIA) as 'randomly sampling a sight range r' and masking the egocentric state via P(s_hat_i, r), but the distribution over r is never stated. The evaluation grid in Section 5.1 is exactly sight-range ratios 0.2, 0.4, 0.6, 0.8, 1.0, 2.0, 3.0, 4.0, 5.0. If r is sampled from a distribution covering these values during offline pretraining, then the message generator and integrator have already been trained on every test sight range. The central claim that TACTIC generalizes to sight ranges 'different from those seen in training' is then only true of the online policy stage, not of the communication mechanism as a whole; at the component level the comparison is interpolation rather than extrapolation. Moreover, the ablations in Figure 9 vary only dataset quality and the two auxiliary losses; no ablation isolates the random-r distribution. Because the four baselines are trained at a fixed sight-range ratio with no analogous random-r augmentation, the reported advantage may reflect simple domain randomization (broader training-data coverage) rather than the contrastive communication objective. This does not disprove the empirical heatmaps, but it is the load-bearing assumption behind the paper's headline interpretation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TACTIC, a two-stage method for cooperative MARL with communication. In an offline stage, a message generator and a message-observation integrator are trained using supervised contrastive losses (GIA and FIA) that align agent-specific integrated representations with egocentric-state embeddings, together with reconstruction and dynamics auxiliary losses. In an online stage, these modules are frozen and inserted into QMIX for policy learning. The method is evaluated on SMACv2 across three maps, team sizes of 5/10/20, training sight-range ratios of 0.2/1/5, and test sight-range ratios from 0.2 to 5, against QMIX, QMIX-Att, NDQ, and TarMAC. The central claim is that TACTIC generalizes better to sight ranges different from those seen during training, particularly under very small or very large sight ranges.","tokens_in":22816,"tokens_out":5090,"duration_ms":48014,"significance":"If the claims hold, TACTIC is a practical contribution: it uses task-agnostic offline data, including random trajectories, to pretrain communication modules that yield better sight-range generalization and faster online learning. The empirical scope is broad (three maps, three team sizes, three training SRRs, nine test SRRs, five seeds, four baselines), and the ablation shows that the two auxiliary losses are jointly important. The main caveats are that the random-sight-range augmentation is not precisely specified and is not ablated, and that no uncertainty quantification is provided, so the strength of the headline generalization claim is not yet established.","major_comments":[{"comment":"The distribution over the randomly sampled sight range r used in P(s_hat_i, r) is never specified. Because the evaluation grid in Section 5.1 is exactly SRR in {0.2, 0.4, ..., 5.0}, if the pretraining r distribution covers this interval, the communication modules have already been trained on all test sight-range ratios, and the claimed 'generalization to sight ranges different from those seen in training' is only true of the online policy stage, not of the communication mechanism as a whole. Please state the sampling distribution, add an ablation that fixes r during FIA (e.g., r equal to the online training SRR) against the random-r version, and ideally evaluate at SRRs outside the pretraining range (e.g., 0.1 and 10) to test extrapolation.","section":"Section 4.1 (FIA) and Section 5.1"},{"comment":"The heatmaps report only mean battle won rates over five seeds, with no standard deviations, standard errors, or significance tests. Since the paper's central claim is comparative (TACTIC outperforms four baselines), the absence of uncertainty quantification makes it impossible to know which numeric differences are meaningful; this is especially relevant for small differences such as those between Figure 9(a) and 9(b). Please report means with error bars or confidence intervals for the main heatmaps and learning curves.","section":"Section 5.2, Figures 5-7"},{"comment":"The ablations vary only dataset quality and the auxiliary losses; there is no ablation isolating the contrastive loss terms (GIA/FIA) or the random-r augmentation. Consequently, the results do not establish that the contrastive objective, rather than simple domain randomization over sight ranges during pretraining, is responsible for the advantage over baselines. Please add ablations with (i) FIA/GIA removed and (ii) FIA with a fixed r, and compare these to the full model.","section":"Section 5.4, Figure 9"},{"comment":"The offline training procedure omits several key hyperparameters: alpha and beta in Eq. (3), W_pos, W_neg, the SupCon temperature tau, and the message embedding dimension. Moreover, the size and composition of the offline dataset D are not fully specified; Section 5.4 mentions 6000 episodes 'per task', but the number of tasks, the collection policy for the main results, and the number of timesteps per episode are not given. Please include a complete hyperparameter and data-collection table.","section":"Section 4.1 and Section 5.1"}],"minor_comments":[{"comment":"The caption says 'deconstruction loss' where the text and Eq. (3) say 'reconstruction loss'; please unify the terminology.","section":"Figure 3 caption"},{"comment":"The text says Figure 8 presents learning curves for 'every combination' of map, agent number, and sight-range ratio, but the figure as printed appears to omit several configurations (notably the 5v5 panels); please correct the text or complete the figure.","section":"Section 5.3, Figure 8"},{"comment":"Reference [30] is cited for QMIX-Att, but the reference is titled 'Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning'; please clarify the relationship or cite the correct source for QMIX-Att.","section":"References"},{"comment":"The color scales in the heatmaps are not described; please state whether all panels share a common color scale, since comparisons across panels currently require reading the numeric cell values.","section":"Figures 5-7"},{"comment":"The phrase 'satisfying performances' would benefit from an explicit operational definition, such as a threshold battle win rate or a comparison rule relative to the training-condition performance.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for AAMAS, and the contribution is clearly distinguished from prior contrastive-communication work. The main risk is the unspecified random-r distribution: if the pretraining distribution already covers the test sight-range ratios, the headline 'generalization' claim needs substantial reframing. I would ask the authors to provide the distribution, add the fixed-r ablation, and report uncertainty estimates; with those changes the central claim could become solid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on TACTIC. The actual contribution is a two-stage recipe: pretrain a message generator/integrator offline with a supervised contrastive loss that aligns integrated observations-plus-messages to each agent's egocentric state, then freeze those modules into QMIX and train the policy online. The random sight-range masking during pretraining is what makes the communication module robust to different observability conditions. That combination is new as far as I know, and it works: across three SMACv2 maps, three team sizes, three training sight ranges, and nine evaluation sight ranges, TACTIC beats QMIX, QMIX-Att, NDQ, and TarMAC by large margins, especially at extreme sight ranges. The evaluation is extensive and the ablations are honest — they show that dropping both auxiliary losses kills the effect, and that even random trajectories for pretraining work.\n\nThe main soft spot is one the authors leave under-specified: the distribution over the random sight range r used in FIA pretraining. If r is sampled from the whole [0.2, 5] interval that later appears in the evaluation grid, then the communication module has been trained on every test sight range, and the 'generalization' claim is really about the online policy stage only. Worse, the baselines are trained at a fixed sight range with no random-r augmentation, so part of the advantage could be plain domain randomization rather than the contrastive objective. That doesn't sink the paper — the method still works and the contrastive alignment is doing something — but the authors need to report the r distribution and ablate it (fixed r vs. sampled r, and maybe held-out ranges). Without that, the headline 'generalizing to sight ranges different from those seen in training' is too strong.\n\nOther issues are smaller: no error bars or confidence intervals on any heatmap or learning curve despite five seeds; no values for alpha, beta, W_pos, W_neg, temperature, or message dimension. And 'task-agnostic' overpromises — the method doesn't use reward in pretraining, but the egocentric-state encoder needs the global state, and everything is evaluated on SMACv2 maps, so 'task-agnostic' is accurate only within the training distribution.\n\nBottom line: this deserves a serious referee. The empirical result is large and the recipe is simple, but the paper needs a revision that specifies the pretraining distribution, adds variance estimates, and ideally an ablation isolating the contrastive loss from the random-r augmentation. If you're in the MARL communication subfield, it's worth reading and citing.","headline":"Solid empirical paper with a real contribution, but the unspecified sight-range sampling distribution undercuts the 'generalization' headline until it's reported and ablated.","tokens_in":23303,"tokens_out":3159,"would_cite":true,"duration_ms":29520,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TACTIC claims that a reward-free contrastive pretraining stage can make multi-agent communication generalize across sight ranges unseen during training, outperforming four baselines on SMACv2.","keywords":["multi-agent reinforcement learning","inter-agent communication","contrastive learning","sight range generalization","partial observability","task-agnostic pretraining","SMACv2","QMIX"],"falsifier":"Train TACTIC with the offline masking procedure restricted to sight-range ratios near the training value, for example only $r=0.2$, and then evaluate at $r=5$; if the battle-win rate in this far-off-diagonal condition drops to the level of the QMIX-Att baseline instead of staying high, the claimed generalization beyond the pretraining distribution is refuted.","tokens_in":1625,"feed_emoji":"📡","tokens_out":1867,"duration_ms":76382,"temperature":0.7,"pith_summary":"TACTIC is a two-stage method for cooperative multi-agent reinforcement learning that aims to make a team's communication robust to changes in how far each agent can see. The paper's central claim is that an offline, reward-free contrastive pretraining stage can teach agents to generate and interpret messages that capture global information, so that the final policy generalizes to sight ranges very different from the one it was trained on. This matters because existing communication methods are reported to degrade sharply when execution-time sight range differs from training-time sight range, forcing retraining for each new visibility condition. On the SMACv2 benchmark, TACTIC is reported to outperform four baselines in cross-sight-range generalization, with the largest margins at extremely small and extremely large sight ranges.","feed_headline":"Pretrained communication lets agents see beyond training sight ranges","feed_subtitle":"A reward-free contrastive stage teaches message passing to generalize across observability, beating four MARL baselines on SMACv2.","key_machinery":"The central object is the egocentric state embedding $\\hat{z}_i$, computed from each agent's global state represented from that agent's perspective, which serves as the alignment target for communication. The mechanism that carries the argument is a pair of supervised contrastive losses: Global Information Alignment (GIA) pulls egocentric-state embeddings of all agents in the same episode and within a timestep window together, and Feature Integration Alignment (FIA) pulls the embedding $z_i$ produced by the message-observation integrator toward the corresponding $\\hat{z}_i$. FIA is the part responsible for sight-range generalization, because the input observation $o_i^r$ is obtained by masking the egocentric state with a randomly sampled sight-range ratio $r$, forcing the message generator to learn to compensate for missing information at many observability levels. Reconstruction and dynamics losses, using a decoder and forward/inverse prediction networks, are auxiliary objectives that keep these embeddings from discarding the content needed to rebuild the egocentric state and to track temporal structure.","core_discovery":"On the paper's own terms, the discovery is that a task-agnostic contrastive objective can replace the usual need to retrain a communication policy for every observability setting. The offline stage trains a message generator and a message-observation integrator by aligning their output embeddings with embeddings of each agent's egocentric global state, using a supervised contrastive loss; a second contrastive loss aligns egocentric embeddings across agents. The generator is exposed to varied sight ranges by randomly masking each egocentric state with a sampled sight-range ratio before producing messages. Two auxiliary losses, reconstruction and forward/inverse dynamics, keep the representations informative and temporally coherent. After this stage, the communication modules are frozen and plugged into a QMIX-style value decomposition, and the resulting policies are evaluated on sight-range ratios from 0.2 to 5. The paper reports that TACTIC maintains usable battle-win rates across the whole tested range even when the training sight-range ratio differs from the test ratio by a factor of 25, while the baselines drop to near-zero win rates in those off-diagonal cases.","pith_inferences":["A testable extension is to train the offline stage with a deliberately narrow distribution of sight-range ratios and then evaluate far outside it; if performance holds, the generalization is true extrapolation, and if not, the result is interpolation across a pretraining-covered range.","The offline stage's reliance on egocentric states derived from the global state means the method needs privileged information during data collection; an extension would be to align instead to a shared reconstructed state available from local observations alone.","The contrastive protocol may be interpreted as learning a compression of the joint state into messages; quantifying the mutual information between messages and the hidden global state could show how much of the communication channel is actually used.","The benchmark evidence is limited to a single environment family, so the task-agnostic claim would be tested more directly by applying the same offline pretraining to a different cooperative domain, such as navigation or robotic swarm tasks."],"forward_implications":["A single trained model can be deployed across environments with different visibility conditions without retraining, provided the execution sight range falls in the range covered by pretraining.","Agents with very narrow sight ranges can maintain coordinated behavior through communication learned offline, reducing the penalty usually associated with limited observability.","Offline communication pretraining accelerates subsequent online policy learning, most clearly when the sight range is small and the number of agents is large.","Because the pretraining stage does not use reward signals, the same communication modules can in principle be reused for different downstream tasks within the same environment.","The task-agnostic pretraining stage means one communication module can replace per-scenario communication policies, making deployment cheaper for systems that must adapt to changing visibility."],"supporting_citations":[{"why":"Supplies the QMIX value-decomposition architecture that TACTIC's online policy stage builds on.","marker":"[15]"},{"why":"Provides the SMACv2 benchmark environment and its scenarios used for all training and evaluation.","marker":"[4]"},{"why":"Defines the TarMAC targeted-message baseline that TACTIC is compared against.","marker":"[2]"},{"why":"Defines the QMIX-Att attention-based communication baseline that TACTIC is compared against.","marker":"[30]"},{"why":"Defines the NDQ communication-minimization baseline used in the comparison.","marker":"[27]"},{"why":"Supplies the supervised contrastive (SupCon) loss that both GIA and FIA are built on.","marker":"[7]"},{"why":"States the sight-range dilemma that motivates the generalization objective.","marker":"[19]"}],"fun_headline_variants":["Contrastive pretraining fixes sight-range dilemma in MARL","Task-agnostic contrastive learning makes agents see further","Pretrained comms adapt to any sight range in MARL","One-size-fits-all communication via contrastive pretraining","TACTIC: communication that survives sight-range shifts"],"cache_read_input_tokens":25344,"weakest_assumption_plain":"The load-bearing premise is that the sight ranges sampled during offline pretraining cover the sight ranges used at evaluation, because the paper never specifies the sampling distribution over $r$, and without that coverage the reported generalization could be interpolation inside an already-seen observability range.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive pretraining fixes sight-range dilemma in MARL","Task-agnostic contrastive learning makes agents see further","Pretrained comms adapt to any sight range in MARL","One-size-fits-all communication via contrastive pretraining","TACTIC: communication that survives sight-range shifts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1311,"prompt_tokens":985,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":244}},"tokens_in":601,"tokens_out":326,"duration_ms":3828,"temperature":1.0,"reasoning_tokens":244,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:13:43.975351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train TACTIC with the offline masking procedure restricted to sight-range ratios near the training value, for example only $r=0.2$, and then evaluate at $r=5$; if the battle-win rate in this far-off-diagonal condition drops to the level of the QMIX-Att baseline instead of staying high, the claimed generalization beyond the pretraining distribution is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the TarMAC targeted-message baseline that TACTIC is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the supervised contrastive (SupCon) loss that both GIA and FIA are built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"States the sight-range dilemma that motivates the generalization objective."}],"review_version":1}