{"id":"e553ef67-a9ea-4e87-a22b-323ab1b3fd17","arxiv_id":"2412.12237","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Augmenting each sampled action with its full symmetry orbit makes finite-sample planning exactly equivariant and speeds up learning on several rotationally symmetric control tasks.","lead":"Continuous control tasks like robotic reaching often look the same after rotations or reflections, but sampling-based planners break that symmetry with finite samples. This paper adds every symmetric copy of each sampled action to the candidate set, making the planner exactly equivariant and helping TD-MPC learn faster on several symmetric control tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The exact-equivariance guarantee is derived for the full symmetry group but implemented only on a finite subgroup; rotations outside the subgroup break the claim, and no error bound is provided.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap: the theory is stated for a continuous group G, while the implementation uses a finite subgroup H. The core mechanism is mathematically sound for a fixed finite group: the augmented orbit set HA is invariant under H, so the argmin transforms correctly under H. The problem is transfer: the tasks' true symmetry is continuous, and the networks and G-augmented sampling are only H-equivariant. For rotations outside H, Equation 5 and Proposition 1 do not apply, so the central claim of exact equivariance is not realized on the reported tasks. This is not an internal inconsistency but a missing argument connecting the implemented discrete symmetry to the continuous problem symmetry. A concrete rotation test would settle whether the discretization is benign: if equivariance error appears for non-subgroup rotations and persists as N grows, the theoretical guarantee is not what drives the results. The empirical observations could still hold, and Figure 11 suggests limited sensitivity to subgroup order on Reacher Hard, so the verdict remains CONDITIONAL rather than REJECT. The reader's verdict already covers this concern, so no adjustment is needed.","tokens_in":24891,"tokens_out":10417,"duration_ms":94848,"concrete_test":"Test-time equivariance check for a trained D8-equivariant TD-MPC on Reacher Hard: for a fixed state s, compute a0 = plan(s) and a_theta = plan(rho_theta s) for theta = 22.5 degrees (not in D8) and theta = 11.25 degrees, using several sample counts N; report ||a_theta - rho_theta a0||. A nonzero error that does not decay with N confirms the finite-subgroup implementation is not equivariant under the continuous task symmetry, contradicting the exact claim. Additionally run a D16 variant and check that the theta = 11.25 degree error shrinks consistently; absence of shrinkage with increasing subgroup order would indicate the gap is not merely a matter of subgroup density.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 6.1 states 'We use discretized subgroups in implementing G-equivariant MLPs with the escnn package'. Consequently the energy, value, dynamics, and policy networks are equivariant/invariant only under H in {D4, D8, C8, icosahedral, octahedral}, not under the continuous symmetry G = O(2), SO(2), SO(3) of the benchmark tasks. The G-augmented sampling proof (Eq. 5 and Proposition 1 in Section 5.3) requires the group action to be exactly that under which E is invariant. For h in G \\ H, the set HA = {h'·a : h' in H} is not invariant under h, so h·argmin_{a in HA} E(s,a) = argmin_{a in HA} E(h·s,a) fails in general; the selection is only H-equivariant. The paper acknowledges discretization but offers no bound on the equivariance error as a function of the distance from h to H, the Lipschitz constant of E, or the sample count. Since the tasks have continuous rotational symmetry, the implemented algorithm is not exactly equivariant under the declared problem symmetry, and the theoretical guarantee does not transfer. The reported 2-3x speedups are therefore attributed to a property that holds only approximately, with the approximation error unquantified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes G-augmented sampling for sampling-based action selection in reinforcement learning and planning. Instead of evaluating only N sampled actions, the method evaluates all group transforms of each sample, i.e., the orbit set GA = {g·a_i}, and then selects the best action from this orbit. The authors show that, for an invariant energy/return function E, this selection rule is exactly G-equivariant for any finite number of samples, provided the argmin is unique or ties are handled equivariantly. They apply the idea to an energy-based coordinate regression task and to an equivariant version of TD-MPC with MPPI planning on several continuous control tasks, reporting that the equivariant version reaches near-optimal performance 2 to 3 times faster in training interactions. The appendix contains proofs of equivariance of the Bellman operator for geometric MDPs, a 'symmetrized policy is always better' argument, and additional ablations.","tokens_in":25163,"tokens_out":11077,"duration_ms":99884,"significance":"The core algebraic observation is elegant and useful: for a fixed finite sample set, enlarging it to the full group orbit makes the argmin-based selection equivariant by construction, with no dependence on the number of samples. If the remaining gaps are closed, the paper would provide a practical recipe for adding symmetry to sampling-based planners such as CEM and MPPI, and the benchmark suite is relevant to the RL and robotics communities. The paper is also honest in reporting ablations on the equivariant components, the choice of finite subgroups, and the warm-up sensitivity. However, the theoretical claims currently outrun the proofs in three load-bearing places: the finite-subgroup implementation, the 'always better' statement, and the treatment of randomness in the sampling procedure.","major_comments":[{"comment":"The theory is developed for the full symmetry group G (O(2), SO(2), SO(3)), but Section 6.1 states that all equivariant MLPs are implemented with finite subgroups H in {D4, D8, C8, icosahedral, octahedral}. For any h in G\\H, the networks are not exactly equivariant and the orbit set HA is not closed under h, so Eq. (5) and Proposition 1 do not hold for h. The paper provides no bound on the equivariance error as a function of how densely H approximates G, the Lipschitz constant of E, or the sample count. Please either prove an approximation guarantee or explicitly restate the theoretical results as H-equivariance rather than G-equivariance.","section":"Section 6.1, Proposition 1 (Section 5.3), Eq. (5)"},{"comment":"The claim in Section 4.3 that the strongly equivariant estimator 'is guaranteed to be a better estimator' and the title of Appendix D.2, 'Equivariant Sampling Is Always Better', are stronger than what the proofs establish. Appendix D.1 proves only the group-averaged inequality ∫_G |H(gx)−Ĥ^G(gx)|dg ≤ ∫_G |H(gx)−Ĥ(gx)|dg, and Appendix D.2 similarly shows that the symmetrized policy is closer on average over group orbits. These results do not imply pointwise improvement for each input x. The main text should either qualify the claim as an improvement in the group-averaged sense or state and prove additional assumptions that yield pointwise dominance.","section":"Section 4.3, Appendices D.1 and D.2"},{"comment":"The proof of Eq. (5) fixes the sample set GA and shows that g·argmin_{a∈GA} E(s,a) equals argmin_{a∈GA} E(g·s,a). In the actual CEM and MPPI procedures described in Sections 4.4 and 5.3, however, the base samples are drawn anew for each state. For a transformed state g·s, the new sample set is only equidistributed with the transformed old set g·GA, not equal to it, so one does not obtain g·a0(s) = a0(g·s) on individual runs unless the random noise itself is transformed as part of an explicitly coupled equivariant map. Please specify the stochastic inputs of the algorithm and prove the exact property that holds; if only distributional (weak) equivariance is achieved, the claim that the procedure 'always preserves equivariance' should be weakened accordingly.","section":"Eq. (5), Section 5.3, Appendix E"},{"comment":"The equivariance identity for argmin also assumes a unique minimizer or an equivariant tie-breaking rule. With multiple minimizers, g·argmin and argmin after transformation may select different elements unless tie-breaking is chosen compatibly with the group action. This assumption should be stated explicitly in Proposition 1 and reflected in the algorithm description, since ties are not impossible with finite sample sets and learned energy functions.","section":"Eq. (5), Appendix E"}],"minor_comments":[{"comment":"There is a typo in 'reinforcment learning' in the sentence about equivariant reinforcement learning methods; it should be 'reinforcement learning'.","section":"Section 1"},{"comment":"In step (3) of the Bellman-operator proof, after substituting a = g^{-1}ā, the transition probability still contains g^{-1}a instead of g^{-1}ā; this appears to be a typographical error in an otherwise valid proof sketch.","section":"Appendix D.3, Eq. (16)-(21)"},{"comment":"The paragraph describing the 'sqrt strategy' and 'linear strategy' is garbled: the sentence 'We use two strategies: sqrt strategy ... we use linear strategy ...' is incomplete and the two strategies are described in an overlapping way. Please rewrite this passage for clarity.","section":"Appendix F.2"},{"comment":"The claim that equivariant methods achieve near-optimal performance '2 to 3 times faster' is not supported by all panels; for example, the PointMass 2D panel shows only a modest advantage for D8-equivariant TD-MPC. Please quantify the speedup across tasks with confidence intervals or specify which panels support the 2-3x statement.","section":"Section 6.2, Figure 6"},{"comment":"Several bibliographic entries contain typos or incomplete metadata, e.g., 'Markov Deision Proesses', 'reinforcment', and the Domshlak entry lacking a year/venue. These should be corrected before publication.","section":"Bibliography"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for the RL/geometric-deep-learning audience and the central orbit-augmentation idea is sound for a fixed sample set. The main reasons for major revision are the mismatch between the continuous-group theory and the finite-subgroup implementation, the overclaimed 'always better' result, and the unaddressed gap between strong equivariance of the argmin over a fixed orbit and the actual stochastic sampling procedure. These are fixable by rewriting the claims and adding precise assumptions, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the core idea is real and worth engaging with. Expanding each sampled action to its group orbit and then selecting the arg-min/max makes action selection exactly equivariant for any finite sample size, provided the score is invariant. That is simple, correct, and demonstrated cleanly in the coordinate regression experiment (Fig. 3). The weak/strong equivariance distinction is a useful framing, and applying orbit averaging to finite-sample action selection in continuous MDPs is a fair novelty, even if the underlying math is classical. The equivariant TD-MPC results show consistent 2-3x sample-efficiency gains on symmetric control tasks, and the appendix ablations (equivariant Q, policy, transition, planning-free, group choice) are more thorough than typical. The authors also honestly note in B.2 that kinematic constraints can limit the benefit.\n\nSoft spots, in proportion. First, the 'always better' claim in D.2 is overstated. The proof shows that the group-averaged error of the symmetrized policy is no larger than the group-averaged error of the original. That is not 'always better' in the usual per-instance sense. The text should say 'better in group-averaged error.' Second, the theory is stated for a compact group G, but the implementation uses finite subgroups (D4, D8, C8, icosahedral, octahedral). The stress-test is right: for h in G\\H, the selected action is not guaranteed to transform correctly, and no error bound is given. The paper acknowledges discretization in Section 6.1 but does not analyze it. This is a real gap between theory and implementation, especially since the tasks have continuous symmetry. It is not fatal—the method still gives exact H-equivariance, and D8 evidently captures enough symmetry to produce the gains—but the claims of exact equivariance should be scoped to the implemented subgroup. Third, the planning experiments do not isolate the G-augmented sampling contribution from the equivariant network components. Figure 9 ablates equivariant Q/pi/transition, but not G-sample on/off. A direct ablation of G-augmented sampling alone would clarify the mechanism's contribution. No code is shipped.\n\nThis paper is for researchers working on equivariant RL/planning and practitioners with symmetric control tasks. It deserves a serious referee: the core idea is correct, the experiments are mostly well done, and the limitations are at least partly acknowledged. I would recommend conditional acceptance: fix the 'always better' overclaim, scope the theory to the implemented subgroup or add an approximation bound, and consider an ablation isolating G-sample.","headline":"G-augmented sampling is a clean, correct mechanism for exact finite-sample equivariance, but the paper overclaims on 'always better' and leaves the finite-subgroup implementation gap unquantified.","tokens_in":25695,"tokens_out":2572,"would_cite":true,"duration_ms":24042,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that expanding each sampled action into its group orbit makes sampling-based action selection exactly equivariant for any finite sample count, and that an equivariant model-based planner built on this operation learns…","keywords":["symmetry","equivariance","action sampling","model-based reinforcement learning","model predictive path integral","continuous control","geometric MDP","G-augmented sampling"],"falsifier":"Take a task with $SO(2)$ or $O(2)$ symmetry, implement $G$-augmented sampling with $D_8$ as in the paper, and measure equivariance error for state rotations by angles that are not multiples of $45^\\circ$ (e.g., $15^\\circ$) using a small fixed sample count; if the selected action fails to rotate by that angle, or the equivariance error jumps above machine precision, the exact-equivariance claim holds only for the discrete subgroup, not for the task's true continuous symmetry.","tokens_in":24633,"feed_emoji":"🎯","tokens_out":9994,"duration_ms":81492,"temperature":0.7,"pith_summary":"Sampling-based action selection in continuous control breaks the exact symmetries of the task whenever the sample set is finite; for example, rotating the input state does not reliably rotate the selected action unless the sampler is exhaustively symmetric. This paper introduces $G$-augmented sampling: after drawing actions from any distribution, each action is expanded into its group orbit $\\{g\\cdot a : g\\in G\\}$, and the best action is chosen from that orbit. Provided the scoring function (energy, Q-value, or return) is $G$-invariant, the chosen action is exactly $G$-equivariant for any number of samples, not just in the infinite-sample limit. The same construction is lifted to multi-step MPPI planning inside an equivariant TD-MPC, and the resulting algorithm reaches near-optimal reward roughly 2 to 3 times faster, in interaction steps, than the non-equivariant baseline on several goal-reaching tasks. A sympathetic reader should care because this closes the gap between equivariant network design and the stochastic optimizers that actually select actions in model-based reinforcement learning.","feed_headline":"G-augmented sampling makes finite action selection exactly equivariant","feed_subtitle":"Expanding each sampled action into its group orbit lets an equivariant TD-MPC reach near-optimal reward 2-3x faster.","key_machinery":"The load-bearing object is the $G$-augmented sample set (the $G$-sample): for each drawn action $a$, the sampler also evaluates every transformed action $g\\cdot a$ with $g$ in the symmetry group $G$, so the candidate set is $GA = \\{g\\cdot a_i : i=1,\\dots,N,\\ g\\in G\\}$. Because the scoring function $E(s,a)$ is $G$-invariant, evaluating $\\arg\\min$ over this orbit-expanded set makes the selection commute with the group action; a rotated state picks the rotated winner. Around this, the paper assembles a geometric MDP formalism: transition dynamics must be $G$-equivariant, reward and value functions $G$-invariant, the policy $G$-equivariant, and the MPPI return functional $G$-invariant, which together make the whole planning procedure equivariant. The other mechanism is the weak/strong equivariance distinction, which diagnoses why finite-sample averages fail and what operation restores the symmetry exactly.","core_discovery":"The central claim is that the stochasticity of sampling, not the learned model, is what breaks symmetry in sampling-based control, and that this can be repaired exactly. The paper defines weak equivariance (holding only after averaging over samples) versus strong equivariance (holding for every finite sample set), and shows that a weakly equivariant estimator can be converted into a strongly equivariant one by symmetrization. Concretely, for action selection $a_0 = \\arg\\min_{a\\in A} E(s,a)$, replacing the sample set $A$ with the $G$-augmented set $GA = \\{g\\cdot a : a\\in A,\\ g\\in G\\}$ makes the map $s\\mapsto a_0$ $G$-equivariant whenever $E$ is $G$-invariant: rotating the state rotates the selected action, for any sample budget. The paper extends this to trajectory planning by requiring $G$-equivariant dynamics, $G$-invariant reward and value networks, a $G$-equivariant policy, and $G$-augmented sampling inside MPPI; Proposition 1 states that the return is $G$-invariant and the $G$-sample procedure is $G$-equivariant. Empirically, an equivariant TD-MPC built from these ingredients matches or beats the non-equivariant TD-MPC and reaches near-optimal reward 2 to 3 times faster (in interaction steps) on PointMass, Reacher, 3D PointMass variants, and MetaWorld Reach. A further claim, proved in the appendix, is that a symmetrized policy or estimator is never worse, in $G$-averaged error, than the unsymmetrized one.","pith_inferences":["The $G$-augmentation recipe is generic: any sampling-based optimizer that scores candidates by an invariant function can be made strongly equivariant by orbit expansion, so the result should transfer to planners beyond the TD-MPC instantiation the paper tests.","A direct test of the finite-subgroup assumption would measure equivariance error for rotations not in the implemented subgroup (e.g., $15^\\circ$ for $D_8$); the paper's guarantees only cover the discrete group elements, so intermediate angles may reveal how much the surrogate matters.","Because symmetrization multiplies the candidate set by $|G|$, the computational cost grows with group size; on large-symmetry problems one might trade exactness for cost by subsampling the orbit, a trade-off the paper does not analyze.","The paper's own ablation shows that on kinematically constrained tasks with global reference frames, equivariance can hurt; the practical benefit likely depends on representing features in the right local frame rather than on equivariance alone."],"forward_implications":["With a $G$-invariant score, $G$-augmented sampling gives exact equivariance for any finite sample count, including a single sample, so the usual infinite-sample asymptotic is not needed.","An equivariant TD-MPC instantiated with discrete subgroups ($D_4$, $D_8$, $C_8$, icosahedral, octahedral) learns near-optimal goal-reaching policies 2 to 3 times faster in interaction steps than non-equivariant TD-MPC on the tested tasks.","In coordinate regression, combining a $G$-invariant energy with $G$-augmented sampling enables spatial extrapolation beyond the training region's convex hull, where the non-equivariant baseline fails.","The symmetrized estimator is provably no worse than the raw estimator in $G$-averaged error, so enforcing symmetry in the sampler does not sacrifice expected accuracy.","Larger discrete subgroups (e.g., $D_{16}$ vs. $D_8$) plateau in performance while increasing cost, suggesting the chosen subgroups already capture the task-relevant symmetry."],"supporting_citations":[{"why":"Supplies the TD-MPC algorithm that the equivariant version modifies and the baseline it is compared against.","marker":"[Hansen et al., 2022]"},{"why":"Defines MPPI, the sampling-based planner whose equivariance the paper analyzes and augments.","marker":"[Williams et al., 2017b]"},{"why":"Defines Implicit Behavior Cloning and the coordinate regression problem used as the first demonstration of $G$-augmented sampling.","marker":"[Florence et al., 2021]"},{"why":"Establishes the symmetry framework for MDPs and the equivariance/invariance conditions that the geometric MDP definition builds on.","marker":"[Ravindran and Barto, 2004]"},{"why":"Provides the result that an optimal policy in a symmetric MDP is $G$-equivariant, which the paper extends from policies to sampling procedures.","marker":"[van der Pol et al., 2020b]"},{"why":"Supplies the equivariant MLP machinery used to implement the $G$-equivariant networks in equivariant TD-MPC.","marker":"[Weiler and Cesa, 2021]"}],"fun_headline_variants":["Sampling-based RL made exactly equivariant by group augmentation","G-augmented sampling fixes symmetry breaking in RL control","Equivariant action sampling: faster RL with exact symmetry","How to make finite action sets perfectly equivariant in RL","Strong equivariance from weak: group orbits repair sampling noise"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the paper's own stated choice (Section 6.1) to replace each task's true continuous symmetry group with a finite discrete subgroup ($D_4$, $D_8$, $C_8$, icosahedral, or octahedral); the exact equivariance guarantee and the empirical gains are proven only for those discrete group elements, and the derivations in Sections 4.4 and 5.3 assume the full group $G$ with no error bound for the discretization.","fun_headline_variants_meta":{"raw":{"variants":["Sampling-based RL made exactly equivariant by group augmentation","G-augmented sampling fixes symmetry breaking in RL control","Equivariant action sampling: faster RL with exact symmetry","How to make finite action sets perfectly equivariant in RL","Strong equivariance from weak: group orbits repair sampling noise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1386,"prompt_tokens":1040,"completion_tokens":346,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":265}},"tokens_in":656,"tokens_out":346,"duration_ms":3570,"temperature":1.0,"reasoning_tokens":265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:21:44.554075+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a task with $SO(2)$ or $O(2)$ symmetry, implement $G$-augmented sampling with $D_8$ as in the paper, and measure equivariance error for state rotations by angles that are not multiples of $45^\\circ$ (e.g., $15^\\circ$) using a small fixed sample count; if the selected action fails to rotate by that angle, or the equivariance error jumps above machine precision, the exact-equivariance claim holds only for the discrete subgroup, not for the task's true continuous symmetry.","supporting_citations":[{"cited_title":"An algebraic approach to abstraction in reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Establishes the symmetry framework for MDPs and the equivariance/invariance conditions that the geometric MDP definition builds on."}],"review_version":1}