{"id":"004e4f37-01a4-4644-829b-bc40ef4247cc","arxiv_id":"2411.10651","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Under a low-dimensional subspace assumption, rescaling informative slices of the sliced-Wasserstein distance reduces to one global constant, so the classical SWD with a tuned learning rate is competitive with specialized variants.","lead":"This paper argues that the plain sliced-Wasserstein distance, a standard way to compare probability distributions, can match fancier variants in machine learning tasks if its learning rate is tuned. The authors show the effect of focusing on informative directions reduces to a single scaling factor, so ordinary hyperparameter search already compensates.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The subspace scaling identity (Thm 4.9) is sound, but the central learning-rate equivalence is only shown in expectation; with L=50 slices the per-iteration ESSF is random, so a fixed learning rate does not generally absorb it.","rationale":"The paper's central claim has two parts: (i) a mathematical identity under exact linear subspace structure (Thm 4.9), which I checked and believe is correctly derived; (ii) an algorithmic translation claiming that because the identity is a global scalar, classical SWD plus LR search matches informative-slice variants. The first part is solid. The second is the load-bearing step for the paper's practical thesis. My concern is not that Theorem 4.9 is wrong, but that it is an identity for population expectations, whereas the optimized objective is a finite-L Monte Carlo estimate. Propositions 4.10 and 4.12 bound the difference between the classic estimator and the ESSF-scaled subspace estimator, but only asymptotically; they do not establish that a single LR absorbs the scaling at L=50. The empirical LR sweep is evidence of competitiveness but not a test of the equivalence mechanism. I therefore recommend keeping the CONDITIONAL verdict: the main theorem appears correct, but the practical claim needs the proposed diagnostic before full acceptance. The Appendix A.3 anisotropic-Gaussian proposition is false, but it is not load-bearing because only the isotropic case is used in the proof; I mention it only as a secondary correctness issue.","tokens_in":71346,"tokens_out":12122,"duration_ms":133745,"concrete_test":"Reproduce the MNIST gradient-flow experiment (50 source, 50 target samples, L=50). At each iteration, compute the realized ESSF(L) from the same 50 random slices and the span of the current minibatch; record its coefficient of variation and the cosine similarity between the classic gradient and the per-slice reweighted 'informative' gradient. Then run the optimization with the loss explicitly corrected by the per-iteration ESSF(L). If the corrected run's final W2 is materially better than the best fixed-LR classic SWD, or if the gradient cosine similarity is far from 1, the 'absorbed into a fixed learning rate' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive step is Section 4.4's claim that the global factor in Theorem 4.9 'can be absorbed into the learning rate.' Theorem 4.9 is exact for the population SWD under Assumption 4.1. For the Monte Carlo estimator actually optimized (L=50), Proposition 4.10 only gives E[ESSF(L)] = C_k/C_d and convergence of the product error to 0; it does not make the realized estimator proportional to the subspace estimator. Indeed the classic MC SWD is (1/L) sum_l ||U^T theta_l||^p W_p^p(theta_k^l#mu_k, theta_k^l#nu_k), while ESSF(L) times the subspace MC SWD is [(1/L) sum_l ||U^T theta_l||^p][(1/L) sum_l W_p^p(...)], equal only in expectation. The difference is nonzero for every finite L and enters the gradient nonlinearly through the sorting in each 1D Wasserstein term. Hence the gradient direction of classic SWD is not a scalar multiple of the 'informative' gradient for finite L; absorbing the scaling into a fixed learning rate is at best an approximation. In minibatch training the effective subspace U also changes each iteration, so even the expected factor can drift. The experiments use a sweeping LR grid, which can hide this mismatch; the paper provides no measurement of ESSF(L) variability in the actual optimization tasks.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the sliced-Wasserstein distance (SWD) for high-dimensional data whose support lies in a low-dimensional linear subspace. Under Assumption 4.1, it proves Theorem 4.9: the ambient SW_p^p is a fixed scalar multiple (C_k/C_d) of the SW_p^p computed in the effective subspace. It introduces a ϕ-weighting framework for rescaling 1D Wasserstein contributions by slice informativeness and argues that the per-slice reweighting simplifies to this single global factor, which can be absorbed into the learning rate in gradient-based learning. The paper reports experiments on gradient flow, color transfer, and deep generative modeling on FFHQ, showing that classical SWD with a well-chosen learning rate matches or outperforms more complex SW variants.","tokens_in":71599,"tokens_out":5636,"duration_ms":55563,"significance":"If the central claim holds, the paper provides a clean explanation of why the classical SWD can be competitive with data-adaptive SW variants: the ill-conditioning associated with uninformative slices is a global scaling phenomenon, rather than a per-slice one. The unifying ϕ-weighting formulation (Section 4.1) is a useful organizing framework. The population-level identity (Theorem 4.9) is exact and the synthetic verification (Section 5.1) is convincing. The paper also ships explicit convergence statements (Propositions 4.10 and 4.12) and an extensive experimental comparison with runtime tables. The main weakness is that the translation from the exact population identity to the finite-slice, minibatch setting used in all practical experiments is not rigorously established; this is the load-bearing step for the headline claim that informative slices come 'for free' via learning-rate search.","major_comments":[{"comment":"The claim that the Effective Subspace Scaling Factor (ESSF) 'can be absorbed into the learning rate' is only established in expectation and asymptotically. For the Monte Carlo estimator actually optimized in the experiments (L=50), the classic SWD is (1/L) Σ_l ||U^T θ_l||^p W_p^p(θ_k^l#µ_k, θ_k^l#ν_k), while the rescaled subspace estimator is [(1/L)Σ_l ||U^T θ_l||^p][(1/L)Σ_l W_p^p(θ_k^l#µ_k, θ_k^l#ν_k)]. These are equal only in expectation; their difference is nonzero for every finite L and enters the gradient nonlinearly through the sorting in each 1D Wasserstein term. Propositions 4.10(2) and 4.12 only give convergence to zero of this error as L→∞. With L=50, no finite-L bound is provided, and the paper reports no measurement of the per-iteration ESSF variability in the actual optimization tasks. The statement in Section 4.4 that the factor 'can be absorbed' is therefore an approximation, not a proven equivalence, and the learning-rate search in the experiments may be compensating for this mismatch. The authors should either provide finite-L gradient-error bounds or explicitly reframe the claim as a heuristic supported by the accompanying empirical study.","section":"§4.4, Proposition 4.10"},{"comment":"In minibatch training, the effective subspace U is not fixed across iterations: Remark 4.11 bounds its dimension by min{2B-1, d}, but the subspace itself changes with each minibatch. Consequently, the expected ESSF, C_k/C_d, is not a single constant throughout optimization; it drifts with U. The paper does not state any assumption under which this drift is negligible, and the experiments use a sweeping learning-rate grid that can absorb per-iteration scaling differences. To support the central claim, the authors should either derive a bound on the drift of the ESSF under minibatch updates or present empirical evidence (e.g., tracking ESSF values over training) that the drift is small relative to the learning-rate sensitivity.","section":"§4.4, Remark 4.11, experiments"},{"comment":"Proposition A.7 states that for any full-rank covariance Σ, f#N(0,Σ) equals the uniform distribution on S^{d-1}, where f(x)=x/||x||. This is false for anisotropic Gaussians: for Σ=diag(σ1^2,...,σd^2), the random direction X/||X|| has a density proportional to ∏ σ_i^{-1} exp(-x_i^2/(2σ_i^2)) on the sphere, which is not constant unless all σ_i are equal. The proof's Step 2 incorrectly identifies Σ^{-1/2}X/||Σ^{-1/2}X|| with X/||X||. The isotropic case used in Proposition A.9 is correct, so this error does not invalidate Theorem 4.9, but the misleading general statement and its proof must be corrected or removed.","section":"Appendix A.3, Proposition A.7"}],"minor_comments":[{"comment":"Remark A.8 contains the typo 'distritbution' and the phrase 'rotationally invariant distritbution'; the sentence should read 'rotationally invariant distribution'.","section":"Appendix A.3, Remark A.8"},{"comment":"The sentence 'The main observation is that all methods perform comparatively, aibeit different learning rates are required for good results' contains the typo 'aibeit' for 'albeit'.","section":"Section 5.2.3"},{"comment":"The appendix heading 'A.4 Relationship between the SWD in Rd and Rk' refers to 'Proposition 4.9' in the proof, but the result is numbered Theorem 4.9 in the main text; the cross-reference should be corrected.","section":"Theorem 4.9 vs. Appendix A.4"},{"comment":"The subsection heading 'A.6.1 Proof of Theorem 4.10' appears before the proof of Proposition 4.10; the theorem number should be 'Proposition 4.10'.","section":"Appendix A.6.1"},{"comment":"Figure 10 is in the Color Transfer section but its caption reads 'Optimal basin plots for Gradient Flow with embedded synthetic datasets.' This caption should be updated to refer to the color transfer results.","section":"Figure 10 caption"},{"comment":"The concentration inequality in Proposition 4.10(3) is stated as P(ϵL < δ) ≥ 1 - e^{-δ^2L/K^2}, but the proof in Appendix A.6 writes a different expression; the constants and exponent should be checked for consistency.","section":"Proposition 4.10(3)"}],"recommendation":"major_revision","confidential_remarks":"The paper's population-level theorem is sound and the experiments are extensive, but the title and abstract promise more than the finite-L analysis delivers. The main revision should focus on making the learning-rate absorption claim precise or explicitly labelling it as a heuristic, and on quantifying the ESSF variability in the actual optimization setups. The incorrect anisotropic-Gaussian claim in Appendix A.3 should be fixed before acceptance, as it is a definite mathematical error that would likely be noticed by readers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: read this if you work on sliced Wasserstein. It does something simple and useful—it shows that under a linear subspace support assumption, the classic SWD is a fixed scalar multiple of the SWD computed in that subspace—and it backs the theory with an honest, broad set of experiments. But the identity is exact only for the population SWD. The step from there to \"absorb the scaling into a fixed learning rate, even with finite slices\" is true only in expectation, and the paper's own appendix contains a false claim about Gaussian projections.\n\nWhat is genuinely new: the phi-weighting formulation is a tidy way to unify several SW variants, and Theorem 4.9 plus the ESSF interpretation gives a real explanation for why high-dimensional SWD underestimates distances. They are careful to disclaim the Gaussian-SW equivalence of Nadjahi et al., so the new piece is the projection change-of-variables argument. The experiments are fair: they sweep learning rates for all methods, use the variants' default hyperparameters, and report runtime and W2. That is credible evidence that a properly tuned classic SW is competitive.\n\nSoft spots, in proportion:\n- The learning-rate absorption claim (Section 4.4) is asserted, not proven. Proposition 4.10 gives E[ESSF]=constant and convergence of the error as L grows, but for L=50, the classic MC SWD is not a scalar multiple of the subspace estimator. The stress-test note is correct: the product of two L-averages differs from the average over slices, and that difference interacts nonlinearly with the sorting inside each 1D Wasserstein term. In minibatch training the effective subspace also changes each step. So \"informative slices for free\" is a reasonable practical approximation, but the paper should either measure ESSF(L) variability during actual optimization or weaken the claim.\n- Appendix A.3, Proposition A.7 states that for any full-rank Gaussian, X/||X|| is uniform on the sphere. That is false unless the covariance is a scalar multiple of identity; the proof's diagonal step is also wrong. It is not load-bearing for Theorem 4.9, but it is an incorrect statement that needs correction.\n- No code or data artifacts are provided. Minor, but a paper whose message is \"tune the learning rate and you are done\" would benefit from releasing the sweep.\n\nThe central theorem holds up, and the empirical story is plausible and useful, but the practical claim is overstated. The paper deserves a serious referee; the right ask is to fix Prop A.7 and either add a direct test of ESSF variability or temper the Section 4.4 language. I would cite the theorem, and I would bring it to the group as a maybe—good counterweight to the \"just sample informative slices\" literature.","headline":"Solid subspace scaling theorem marred by an overstated learning-rate absorption claim and a false appendix proposition; still worth refereeing.","tokens_in":72168,"tokens_out":3644,"would_cite":true,"duration_ms":46658,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that classical Sliced-Wasserstein, with a properly tuned learning rate, can match or beat more complex informative-slice variants in common learning tasks.","keywords":["Sliced-Wasserstein distance","informative slices","learning rate","low-dimensional subspace","optimal transport","gradient flow","generative modeling"],"falsifier":"One concrete check: take two distributions on a common $k$-dimensional subspace and verify the empirical ratio of ambient to subspace sliced-Wasserstein approaches $C_k/C_d$ as the number of slices grows; the broader learning claim would be refuted by a common task, after a full learning-rate sweep, where plain SWD falls materially short of a tuned informative-slice variant.","tokens_in":71122,"feed_emoji":"📐","tokens_out":7709,"duration_ms":73402,"temperature":0.7,"pith_summary":"The paper tries to establish that the standard Sliced-Wasserstein distance (SWD), often dismissed in high dimensions because random projections are uninformative, already contains the information those projections miss: when two distributions live in a common low-dimensional subspace, the ambient SWD is exactly a fixed scalar multiple of the SWD computed inside that subspace. Because the scalar depends only on ambient and effective dimensions, reweighting every slice by its informativeness reduces to multiplying the whole distance by one number, which in gradient-based learning is absorbed by the standard learning-rate search. The authors propose a $\\phi$-weighting framework that unifies classical SWD with Max-SW, energy-based, Markovian, and random-path variants, then show empirically across gradient flow, color transfer, and generative modeling on FFHQ latents that a properly configured classical SWD often matches or surpasses these variants. A sympathetic reader would care because it means much of the added complexity of informative-slice SW variants may be unnecessary for common tasks: the plain distance plus a tuned step size can be both simpler and cheaper.","feed_headline":"Tune the rate and plain Sliced-Wasserstein matches the fancy variants","feed_subtitle":"One global rescaling factor captures what informative-slice methods work to find, so a learning-rate sweep may be enough.","key_machinery":"The load-bearing object is the $\\phi$-weighting formulation $gSW_p^p(\\mu,\\nu;\\sigma,\\rho_\\phi) = \\int_{S^{d-1}} \\rho_\\phi(\\phi(\\theta))\\, W_p^p(\\theta_\\#\\mu, \\theta_\\#\\nu)\\, d\\sigma(\\theta)$, which reweights each projected 1D Wasserstein by a function of slice informativeness instead of resampling slices. With the ES-aligned informativeness $\\phi_U(\\theta) = \\lVert U^\\top \\theta \\rVert$ and the reciprocal weighting $\\rho_\\phi(r) = r^{-p}$, Proposition 4.7 shows each slice contribution satisfies $W_p^p(\\theta_\\#\\mu_d, \\theta_\\#\\nu_d) = \\lVert U^\\top \\theta \\rVert^p W_p^p(\\theta_{k\\#}\\mu_k, \\theta_{k\\#}\\nu_k)$, so the reweighted distance equals the SWD in the $k$-dimensional effective subspace. Theorem 4.9 then identifies the expectation of $\\lVert U^\\top \\theta \\rVert^p$ with the ratio $C_k/C_d$ of two Gamma-function constants, turning per-slice rescaling into one global scalar. Proposition 4.10 extends this to finite samples and Monte Carlo slice estimators, with variance $O(1/L)$. The machinery's role is to convert the concentration-of-measure objection, that most random slices are nearly orthogonal to the data subspace, into a harmless multiplicative constant that learning-rate tuning already handles.","core_discovery":"Under Assumption 4.1, where both measures are supported on the same $k$-dimensional linear subspace $V_k$ of $\\mathbb{R}^d$, the $p$-th power of the sliced-Wasserstein distance obeys $SW_p^p(\\mu_d, \\nu_d) = (C_k/C_d)\\, SW_p^p(\\mu_k, \\nu_k)$, with $C_d = 2^{p/2}\\Gamma((d+p)/2)/\\Gamma(d/2)$ and $C_k$ defined analogously. In other words, the information lost by averaging over all random projections of high-dimensional data is not a per-slice accident: it is a single dimension-dependent scaling constant. Defining slice informativeness as $\\phi_U(\\theta) = \\lVert U^\\top \\theta \\rVert$, the alignment of a direction $\\theta$ with the effective subspace, rescaling each one-dimensional Wasserstein contribution by $\\phi_U(\\theta)^{-p}$ exactly cancels the implicit downweighting of misaligned slices, and in expectation this per-slice reweighting collapses to the global factor $C_d/C_k$. The paper's central claim is that in gradient-based learning this global factor is absorbed by learning-rate selection, so classical SWD with a properly chosen step size obtains informative slices for free and can perform competitively with more complex SW variants in common learning tasks.","pith_inferences":["A testable extension is to compare classical SWD against informative-slice variants on data whose support is a curved manifold rather than a linear subspace, where per-slice reweighting may not collapse to a global scalar and the advantage of adaptive slicing could reappear.","The assumption of a shared common subspace for both distributions is stronger than the manifold hypothesis; if source and target occupy different low-dimensional subspaces, the global-factor argument may fail, and explicit subspace-aware methods would retain an edge.","Because the scaling factor depends only on ambient and effective dimension, one could estimate the effective dimension from minibatches and predict the optimal learning-rate shift across tasks, making the learning-rate search more principled.","If the claim is right, benchmark protocol changes: future SW-variant evaluations should include a well-tuned classical SWD baseline, otherwise apparent gains may reflect learning-rate mismatch rather than a genuine advantage of the new method."],"forward_implications":["If the central claim is correct, the classical SWD needs no data-dependent slicing distribution to be competitive in common tasks: a learning-rate sweep over the plain distance is sufficient.","The effective-dimensionality dependence of the scaling factor means the same classical SWD automatically adapts to data with lower intrinsic dimension, without explicitly estimating the subspace.","Nearly all informative-slice SW variants, including Max-SW, EBSW, Markovian SW, and RPSW, are recovered as special cases of the $\\phi$-weighting framework, so the metric properties of the classical distance are preserved when the global rescaling is applied.","Because the global factor can be absorbed into a learning rate, comparisons among SW variants in learning tasks should control for learning-rate search; reported gaps can shrink or reverse when the classical baseline is tuned.","Proposition 4.10 gives convergence rates for the gap between the ambient Monte Carlo SWD and the scaled subspace SWD, so users know how many slices are needed for the scalar-factor reduction to hold."],"supporting_citations":[{"why":"Defines the Sliced-Wasserstein distance and its efficient one-dimensional closed-form computation, the object under study.","marker":"(Bonneel et al., 2015)"},{"why":"Supplies the concentration-of-measure bound showing most random slices are nearly orthogonal to a fixed direction, motivating the informativeness problem.","marker":"(Kolouri et al., 2019)"},{"why":"Establishes the Gaussian-slicing equivalence with a dimensionality-dependent constant, which the proof of Theorem 4.9 uses to derive the ratio $C_k/C_d$.","marker":"(Nadjahi et al., 2021)"},{"why":"Provides Max-SW, a central informative-slice baseline that the $\\phi$-framework recovers and the experiments compare against.","marker":"(Deshpande et al., 2019)"},{"why":"Provides EBSW, an energy-based slicing variant used both as a special case of the framework and as an experimental baseline.","marker":"(Nguyen and Ho, 2024)"},{"why":"Provides RPSW, a random-path projection variant compared in the experiments and recovered in the $\\phi$-weighting unification.","marker":"(Nguyen et al., 2024b)"},{"why":"Gives intrinsic dimension estimates for image datasets, cited as empirical support for the low-dimensional subspace assumption.","marker":"(Pope et al., 2021)"}],"fun_headline_variants":["Plain SWD matches fancy slices when learning rate is tuned","Forget informative slices: rescale once, tune the rate","Learning rate is the hidden informative slice","All slices equally informative via one global scaling","Tune the rate, not the slices: SWD does it all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes the two distributions sit on the same low-dimensional linear subspace, and then presumes the resulting rescaling factor stays roughly constant across minibatches so a single learning rate can absorb it.","fun_headline_variants_meta":{"raw":{"variants":["Plain SWD matches fancy slices when learning rate is tuned","Forget informative slices: rescale once, tune the rate","Learning rate is the hidden informative slice","All slices equally informative via one global scaling","Tune the rate, not the slices: SWD does it all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2899,"prompt_tokens":1093,"completion_tokens":1806,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":709,"completion_tokens_details":{"reasoning_tokens":1728}},"tokens_in":709,"tokens_out":1806,"duration_ms":14076,"temperature":1.0,"reasoning_tokens":1728,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:28:18.664123+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check: take two distributions on a common $k$-dimensional subspace and verify the empirical ratio of ambient to subspace sliced-Wasserstein approaches $C_k/C_d$ as the number of slices grows; the broader learning claim would be refuted by a common task, after a full learning-rate sweep, where plain SWD falls materially short of a tuned informative-slice variant.","supporting_citations":[],"review_version":1}