{"id":"6f7f6e8b-c76b-49e1-9c2a-07c93dd1dfce","arxiv_id":"2509.08257","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Symmetry-guided data augmentation and a symmetry-aware discriminator improve sample efficiency in multi-agent inverse reinforcement learning.","lead":"This paper shows that using symmetry, like rotations and reflections, in multi-agent inverse reinforcement learning lets algorithms recover better reward functions from fewer expert demonstrations. The authors build a framework that augments demonstrations with symmetric transformations and validate it on simulated and physical robot tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 2 is not proven: the appendix proof uses an inverted concentration bound and an internally contradictory support-term argument, so the paper's central theoretical claim is unestablished.","rationale":"Good-faith reading: the paper proposes a sensible framework (SGDA+SAD) and reports consistent gains. But the theoretical contribution is Proposition 2, and the appendix proof is not merely missing details; it contains an inverted concentration bound and a contradictory support-term argument. I checked whether the theorem might still be true under exact G-invariance: the second-term comparison can probably be repaired with C/sqrt(D), and the support term is likely harmless, so the idea may survive a corrected proof. However, as written the central claim is unsupported. I did not base this on disagreement with mainstream practice; exact G-invariance is an explicit idealized assumption, but it is also unvalidated, so the empirical section does not rescue the theory. The experiments' lack of error bars and single real-world trials are secondary. A formal re-derivation or counterexample is the direct test.","tokens_in":13896,"tokens_out":12931,"duration_ms":145178,"concrete_test":"Formalize Proposition 2 with explicit quantifiers, e.g. U(hat B)=sup_{hat r in R_hat B}|r-hat r| and U(hat B^G)=sup_{hat r^G in R_hat B^G}|r-hat r^G|, and re-derive the claim using a standard Bernstein bound |P-hat P|_1 <= C sqrt(log(1/delta)/D) for D>0 and 1 otherwise. If the inequality cannot be proven, or fails on a small D4-invariant MDP (e.g., a two-state cyclic game with one expert trajectory), the proof gap is fatal. If it can be proven, the theory should be credited with a corrected appendix.","verdict_should_be":"REJECT","load_bearing_attack":"Section IV-C's Proposition 2 is the headline claim, but the appendix proof is invalid as written. (1) It cites [34,35] for |P(s'|s,a)-hat P(s'|s,a)| <= C sqrt(D(s,a)) < 1. This is the wrong scaling: standard concentration bounds give O(1/sqrt(D)), not O(sqrt(D)). With the printed C sqrt(D), the later step '1 - C sqrt(D_G) > 0' is not a valid concentration argument, so the second term of delta is not shown nonnegative. (2) For the support term, when (s,a) in tau^G_E \\ tau_E the proof asserts hat pi_E(a|s)>0 and then derives hat pi^G_E(a|s)=0, contradicting the fact that (s,a) lies in the augmented dataset. Under exact G-invariance this term is actually 0, not positive; the proof's claimed route to delta >= 0 does not work. (3) The proof introduces delta as a difference of actual errors but manipulates it as a difference of upper bounds, and the quantifiers of Proposition 2 (which hat r in R_hatB, which hat r^G, sup or inf over feasible sets) are never stated. Even granting exact G-invariance, the central inequality is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to improve sample efficiency in multi-agent inverse reinforcement learning (MIRL) by exploiting rotational/reflectional symmetry. It extends the feasible-reward-set characterization of Metelli et al. to Markov games (Lemma 1), derives an error-propagation bound for the empirical MIRL problem (Proposition 1), and claims that augmenting expert demonstrations by the group action reduces the worst-case reward-recovery error (Proposition 2). Based on this theory, the authors introduce a Symmetry-Guided Framework (SGF) with a demonstration augmenter (SGDA) and a symmetry-aware discriminator (SAD), and evaluate it on three simulated multi-agent tasks and two physical robot tasks.","tokens_in":14072,"tokens_out":6517,"duration_ms":86499,"significance":"If Proposition 2 were established, the paper would make a useful contribution: it would provide a principled justification for symmetry-based data augmentation in MIRL, it is framework-agnostic, and it introduces no free parameters. The experimental results are consistently in favor of the proposed SGF and include a physical-robot validation, which is a strength. However, the central theoretical claim is not proven as written. Lemma 1 and Proposition 1 are clean extensions of known results, but Proposition 2--the headline contribution--rests on a proof with multiple load-bearing errors. Because the theoretical contribution is the paper's main novelty, these errors are significant despite the promising empirical results.","major_comments":[{"comment":"The proof cites [34,35] for the concentration bound |P - P_hat| <= C sqrt(D(s,a)) < 1. This scaling is inverted: standard concentration bounds for empirical transition probabilities decay as O(1/sqrt(D(s,a))), not O(sqrt(D(s,a))). Consequently, the subsequent step '1 - C sqrt(D_G(s,a)) > 0' is not a valid concentration argument. The second term of delta is therefore not shown nonnegative as claimed.","section":"Appendix, proof of Proposition 2"},{"comment":"The support-term argument is internally contradictory. When (s,a) in tau^G_E \\ tau_E, the proof first asserts hat{pi}_E(a|s) > 0 and then derives hat{pi}^G_E(a|s) = 0. But (s,a) is in the augmented dataset by assumption, so hat{pi}^G_E(a|s) > 0. Moreover, under exact G-invariance, if pi_E(a|s)=0 then no transformed sample can create (s,a) in tau^G_E, so the first term is actually zero in all relevant cases. The proof's sign analysis of the first term is not valid.","section":"Appendix, proof of Proposition 2, support term"},{"comment":"The proposition is not well-quantified. It compares |r(s,a)-hat r(s,a)|_U and |r(s,a)-hat r^G(s,a)|_U, but it does not state over which r, hat r, and hat r^G the comparison is made, nor whether U denotes the supremum or infimum of the corresponding upper bounds. In the proof, delta is introduced as a difference of actual errors but is then manipulated as a difference of upper bounds from Proposition 1. Without fixed quantifiers, the claimed inequality cannot be checked.","section":"Section IV-C, Proposition 2"},{"comment":"The theoretical guarantee relies on the exact G-invariance of both the environment transition P and the expert policy pi_E under the dihedral group D_n. The SGDA then treats group-transformed samples as valid expert data. In the experimental environments, including the physical robots, symmetry is only approximate; the paper does not measure the degree of symmetry violation or show that Proposition 2 degrades gracefully. As stated, the bound need not hold under approximate symmetry, so the experiments do not fully validate the theoretical claim.","section":"Section IV-C, Definition 1 and Section V-A"}],"minor_comments":[{"comment":"The augmented dataset definition contains typos: it should be {(g s_j, g a_j, g s'_j)} with (s_j,a_j,s'_j) in tau_E, not {g a} and '(s_j,a_j,s_j)'.","section":"Definition 2"},{"comment":"The parameter zeta is said to lie in R^{S x S}_{>=0}, but it should be indexed by state-action pairs, i.e., R^{S x A}_{>=0}, consistent with Lemma 1.","section":"Lemma 2"},{"comment":"The paper alternates between hat B_G and hat B^G for the symmetry-guided MIRL problem; please unify the notation.","section":"Notation"},{"comment":"The phrase 'choose V = hat V' is stated backwards; the intended construction is choose hat V = V and hat zeta = zeta I{pi_E(a|s)=0}. Please rephrase.","section":"Appendix, proof of Proposition 1"},{"comment":"There is a typo: 'D(s,a) is the number of of (s,a) in tau' should read 'number of occurrences of (s,a)'.","section":"Appendix, concentration bound"},{"comment":"The reward visualization would benefit from a precise description of the action grid and the state/action normalization used; currently it is difficult to interpret the scale and orientation of the recovered rewards.","section":"Section VI-C and Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The technical objections from the stress-test land: the proof of Proposition 2 is invalid as written, and the exact-invariance assumption is not probed in the experiments. I nonetheless recommend major revision rather than rejection because the claimed result is plausible--with more data the empirical transition model should concentrate faster--and the proof appears repairable with a correct concentration bound and a corrected support-term argument. If the authors cannot supply a rigorous proof of Proposition 2, the paper should not be accepted in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper has a good empirical story, but the theory does not hold up. The main claim—that symmetry provably improves reward recovery in multi-agent IRL—is not proven.\n\nWhat is actually new: the formalization of a G-invariant Markov game for MIRL and the attempt to extend Metelli et al.'s error-propagation result to the symmetry-augmented problem. The SGF framework itself (SGDA + SAD) is a straightforward adaptation of symmetry augmentation from MARL to adversarial IRL, but it is cleanly presented. The experiments are the real contribution: three continuous cooperative tasks, five seeds, consistent improvement over MA-GAIL and MA-AIRL, an ablation isolating the discriminator, and a physical robot validation. That is honest work.\n\nThe soft spot is Proposition 2. The appendix proof is flawed in a load-bearing way. It cites a concentration bound with inverted scaling: |P-P_hat| <= C sqrt(D(s,a)). Standard results give O(1/sqrt(D)). The step '1 - C sqrt(D_G) > 0' is therefore nonsense for large D_G. And the support-term case in the proof contradicts itself: if (s,a) is in the augmented dataset but not the original, exact G-invariance implies its pre-image is in the original dataset, which would make the true expert policy put positive mass on (s,a), contradicting the assumption pi_E(a|s)=0. So the term is actually 0, not positive. The quantifiers of Proposition 2 are also left vague. This is not a minor gap; the central inequality is simply not derived.\n\nA related soft spot is the exact-invariance assumption, never validated on these tasks. Real systems are only approximately symmetric. The missing error bars in Figure 3 and the single-value real-world table are minor by comparison.\n\nWho this is for: researchers working on sample-efficient IRL or symmetry in multi-agent learning. The empirical framework is worth trying even if the proof is wrong. A serious referee should ask for a corrected proof or a weakening of the claim, and the paper should not be accepted as is. But it deserves referee time.","headline":"A useful empirical framework for symmetry-guided multi-agent IRL, but the headline theoretical guarantee is not proven.","tokens_in":14688,"tokens_out":3394,"would_cite":false,"duration_ms":40298,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that using the inherent symmetry of multi-robot systems lets inverse reinforcement learning recover more accurate reward functions from fewer expert demonstrations, and packages the result into a plug-in training framework","keywords":["multi-agent inverse reinforcement learning","sample efficiency","symmetry","equivariance and invariance","dihedral group","generative adversarial imitation learning","multi-robot systems","reward recovery"],"falsifier":"In a synthetic Markov game with known ground-truth reward and controlled transition symmetry, perturb the transition by increasing magnitudes that break the G-invariance; if the symmetry-augmented algorithm's reward-recovery error becomes larger than the unaugmented baseline at small perturbations, the practical premise of Proposition 2 is experimentally contradicted.","tokens_in":13682,"feed_emoji":"🤖","tokens_out":5251,"duration_ms":61846,"temperature":0.7,"pith_summary":"Multi-agent inverse reinforcement learning normally needs many expert demonstrations to infer a reward function. This paper claims that if the underlying system is symmetric—its dynamics and expert policy unchanged by rotations and reflections—then augmenting the demonstrations with those transformations provably recovers a more accurate reward function than training on the raw demonstrations alone. The formal result, Proposition 2, compares the worst-case error of the two problems and shows the symmetry-guided problem's upper bound is no larger. The authors turn this into a framework, SGF, that augments demonstrations and adds a symmetry-aware discriminator, and they show it improves sample efficiency in simulated cooperative tasks and on physical robots.","feed_headline":"Symmetry makes learned rewards more accurate with fewer demos","feed_subtitle":"Turning each expert demo into many symmetric variants tightens the recovered reward, per a new bound and robot tests.","key_machinery":"The load-bearing object is the G-invariant Markov game without reward (Definition 1), i.e. a Markov game whose transition P and expert policy pi_E satisfy P(s'|s,a) = P(L_g[s']|L_g[s],K_g[a]) and pi_i(a_i|s) = pi_i(K_g[a_i]|L_g[s]) for every transformation g. The proof uses the feasible-reward-set characterization of Lemma 1 to convert symmetry into a comparison of empirical transition estimates. The framework then operationalizes the theory with the Symmetry-Guided Demonstration Augmenter (SGDA), which applies dihedral-group rotations and reflections D_n to expert and generated samples, and a Symmetry-Aware Discriminator (SAD) that adds a symmetry-augmented cross-entropy term to the adversa","core_discovery":"The paper's central claim is Proposition 2. For a Markov game without reward whose transition function and expert policy are invariant under a symmetry group G, define the symmetry-guided multi-agent inverse reinforcement learning problem from the augmented demonstration set obtained by applying every g in G to each sample. Then the worst-case error in recovering any feasible reward is no larger with this augmentation than without it: |r(s,a) - r_hat(s,a)|_U >= |r(s,a) - r_hat^G(s,a)|_U. The proof combines a characterization of the feasible reward set (Lemma 1) with an error-propagation bound (Proposition 1) and transition-estimation concentration arguments. The authors conclude that leverag","pith_inferences":["A natural extension the paper does not test: estimate the degree of G-violation per environment and let the augmentation weight adapt, so approximately symmetric systems are not hurt by invalid synthetic samples.","The bound is worst-case; testing average-case reward error on known-symmetric environments with ground-truth rewards would show whether the practical gains come from tighter bounds or from added regularization.","The same reasoning could apply to single-agent inverse reinforcement learning with structured transformations such as translations or reflections, since the feasible-reward-set argument is per state-action pair and does not require multiple agents.","One could combine SGDA with partial-symmetry methods to handle realistic robot asymmetries; if approximate symmetry weakens the bound's premise, a partial-symmetry variant would be the next experiment."],"forward_implications":["On Rendezvous, Pursuit, and Vicsek, S-MA-AIRL and S-MA-GAIL outperform their base algorithms at every tested demonstration count, with the largest gaps at the smallest counts.","Fewer demonstrations suffice: symmetry-guided runs reach rewards comparable to baselines trained on far more demonstrations.","The Symmetry-Aware Discriminator is load-bearing: removing it and using a normal discriminator consistently lowers convergence rewards in the ablation study.","Performance degrades with agent count for baselines, but SGF largely holds up, maintaining stronger rewards for 5 to 20 agents.","Physical robots confirm the effect: in real-world Rendezvous and Pursuit tasks, S-MA-AIRL completes tasks faster than MA-AIRL."],"supporting_citations":[{"why":"Supplies the feasible-reward-set characterization and the error-propagation bound that Lemma 1 and Proposition 1 extend to multi-agent settings.","marker":"[30]"},{"why":"Demonstrates in multi-agent reinforcement learning that rotational symmetry with data augmentation improves sample efficiency, the prior the paper turns into an MIRL theory.","marker":"[17]"},{"why":"Defines the MA-AIRL objective and discriminator structure that SGF wraps with the symmetry-aware loss.","marker":"[20]"},{"why":"Defines the multi-agent generative adversarial imitation learning objective used as the other baseline and target of the framework.","marker":"[19]"},{"why":"Provides the transition-estimation error bound used in the proof of Proposition 2 to compare augmented and non-augmented error.","marker":"[34]"},{"why":"Supplies the additional concentration bound the proof cites for transition-error estimation.","marker":"[35]"},{"why":"Trains the expert policies that generate demonstrations in the rendezvous and pursuit experiments.","marker":"[31]"}],"fun_headline_variants":["Symmetry boosts reward accuracy with fewer demos","Leverage symmetry for more accurate rewards from less data","Symmetry cuts demo needs in multi-agent reward learning","Richer rewards via symmetry: less data, better accuracy","Symmetry-guided rewards: more precise from fewer demos"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The guarantee assumes the environment's dynamics and the expert's policy are exactly invariant under the chosen rotations and reflections; real multi-robot systems are only approximately symmetric, so augmented demonstrations may come from a wrong distribution and Proposition 2's bound need not hold.","fun_headline_variants_meta":{"raw":{"variants":["Symmetry boosts reward accuracy with fewer demos","Leverage symmetry for more accurate rewards from less data","Symmetry cuts demo needs in multi-agent reward learning","Richer rewards via symmetry: less data, better accuracy","Symmetry-guided rewards: more precise from fewer demos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000147,"raw_usage":{"total_tokens":1006,"prompt_tokens":709,"completion_tokens":297,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":233}},"tokens_in":453,"tokens_out":297,"duration_ms":3864,"temperature":1.0,"reasoning_tokens":233,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T20:55:45.234280+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In a synthetic Markov game with known ground-truth reward and controlled transition symmetry, perturb the transition by increasing magnitudes that break the G-invariance; if the symmetry-augmented algorithm's reward-recovery error becomes larger than the unaugmented baseline at small perturbations, the practical premise of Proposition 2 is experimentally contradicted.","supporting_citations":[],"review_version":1}