{"id":"dfb3913e-3d48-483e-b35d-751db4d6efd5","arxiv_id":"2506.09454","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Taylor-approximated softmax losses RG and RG are claimed DCG-consistent and, under ALS optimization, reach comparable or better ranking quality than softmax with faster convergence.","lead":"This paper derives two new ranking loss functions as quadratic approximations of the softmax loss, and shows they can be trained with fast closed-form updates. In recommender-system tests the new losses match or beat softmax ranking quality while converging faster.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Experiments replace the derived W,V coefficients with tuned α,β (App. B.2), so the reported gains validate a tuned squared-loss family rather than the exact RG2/RG× objectives whose DCG-consistency is proved.","rationale":"I considered the reader's weakest assumptions (rank-K Bayes optimality, Taylor expansion at zero). Those are real but secondary: rank-K misspecification is a standard limitation of all MF consistency analyses, and DCG-consistency for RG losses is proved directly from the squared form, not from fidelity of the Taylor expansion. The mismatch between the theoretically derived coefficients and the tuned coefficients actually implemented is more load-bearing because it breaks the link between the formal guarantees and the empirical evidence, and it is explicitly admitted in App. B.2. A single rerun with the exact Eq. (32) coefficients would settle whether the derived objectives retain the reported performance. If they do, the conditional acceptance stands with the concern addressed; if they do not, the paper should be revised to either present the tuned variant as a heuristic extension or add an ablation showing the tuned coefficients approximate the derived ones. I therefore agree partially with the reader: the same tuning issue is in their rationale, but it is not their identified weakest assumption. The appropriate verdict remains CONDITIONAL.","tokens_in":25644,"tokens_out":17783,"duration_ms":194948,"concrete_test":"Rerun the MovieLens, Electronics, and Steam experiments exactly as in Table 3 but with W and V fixed to the derived values in Eq. (32) (W_x=|I_x| and V_x=|I_x| for RG2/RG×; S_{x,y}=r_{x,y}N/|I_x|-1), tuning only the regularization λ over the reported grid. Compare NDCG@10 and MRR@10 with the tuned α,β results, and also measure the ranking overlap between the two solutions. If the exact-coefficient results are within experimental noise or better, the concern is mitigated; if they are materially worse, the reported method is a tuned variant outside the proven consistent family and the central claim should be re-scoped.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theoretical core (Sec. 4.2) proves DCG-consistency for L_RG2 and L_RG× with the specific coefficients W_x=|I_x| and target S_{x,y}=r_{x,y}N/|I_x|-1 (Eqs. 16-18, 32). The proof of Theorem 12 and the Bregman-divergence argument depend on these exact weights: the link functions g2(o)=2|I_x|o and g×(o)=2|I_x|(I-11^T/N)o are inverse order-preserving, and the paper explicitly warns that assigning different weights to positive and negative samples 'may disrupt this property and hence affect DCG-consistency' (Sec. 4.2). However, App. B.2 states: 'Considering the operability of experiments, the coefficients W and V in equations are replaced by hyperparameters α, β.' Thus the numbers in Tables 3-4 and Figure 1 are for a hyperparameter-tuned objective, not for the objective analyzed in Theorem 12. If α and β are tuned separately for positive and negative interactions, the minimizer of the weighted squared loss is not necessarily a monotone transform of E[r|x,y], so the DCG-consistency proof no longer applies. If they are uniform constants, then the paper's claim that the derived weights are responsible for the performance is untested. Either way, the central empirical assertion — that the proposed theoretically motivated RG losses match or beat softmax — is not actually evaluated; what is evaluated is a tuned member of the WRMF family, whose strong ranking performance is already known. This gap is load-bearing because it connects the consistency/approximation theory to the empirical results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two squared-form surrogate losses, RG² and RG×, obtained by a second-order Taylor expansion of the softmax and sampled-softmax losses at the zero logit vector. The authors show that these losses can be written as weighted squared losses with weights determined by |I_x| and the sampling size n, thereby connecting non-sampling losses with sampling-based softmax variants. They prove DCG-consistency of the proposed losses via a Bregman-divergence argument, provide a generalization bound for the RG losses instantiated with matrix factorization, and state an ALS convergence-rate comparison. Experiments on MovieLens, Amazon Electronics, Steam, and Wikipedia compare RG² and RG× against BPR, BCE, SSM, Sparsemax, UIB, SML, WRMF, and softmax, reporting that the RG losses match or exceed softmax ranking performance with substantially faster convergence under ALS. The main theoretical claims rely on the assumption that the Bayes-optimal scoring function lies in the low-rank hypothesis space, and the appendix states that the experimental implementation replaces the derived weights W and V by tuned hyperparameters α and β.","tokens_in":26031,"tokens_out":12251,"duration_ms":125136,"significance":"If the claims are fully substantiated, the paper offers a valuable conceptual bridge: it derives WSL-type weights from a Taylor expansion of the softmax loss, provides a theoretical explanation for the empirical success of weighted squared losses in ranking, and connects sampling and non-sampling loss families through the sampling size n. The DCG-consistency framework and the generalization bound are useful additions to the ranking theory literature, and the experiments cover diverse datasets and baselines. However, the current empirical validation is weakened by the replacement of the derived weights with tuned hyperparameters, so the reported gains do not directly test the theoretically motivated objective. The formal consistency and generalization results also depend on assumptions (low-rank Bayes optimality, accuracy of the Taylor expansion) that are not validated on the trained models. These gaps are fixable but require substantive additional work, so the paper is not yet ready for acceptance.","major_comments":[{"comment":"The experiments evaluate a different objective from the one whose theory is established. Appendix B.2 states that 'the coefficients W and V in equations are replaced by hyperparameters α, β,' while Eq. (32) defines W and V as |I_x|, |I_x|/N, or w_xy. The DCG-consistency result in Theorem 12 and the Bregman argument in Section 4.2 depend on these exact weights: the link functions g2 and g× are inverse order-preserving only for uniform weighting, and the paper itself warns that assigning different weights to positive and negative samples 'may disrupt this property and hence affect DCG-consistency.' Consequently, Tables 3-4 and Figure 1 validate a tuned weighted-squared-loss family rather than the proposed RG²/RG× objectives. If α and β are tuned separately for positives and negatives, the consistency proof no longer applies; if they are uniform constants, the claimed benefit of the derived weights is untested. The central empirical claim must be re-run with the exact derived W and V, or the paper must explicitly reframe the experiments as evaluating a separate tuned surrogate family.","section":"Section 5 and Appendix B.2"},{"comment":"The consistency and generalization results require f_B(x,y)=E[r|x,y] to lie in the hypothesis space H, as stated after Eq. (25). For rank-64 matrix factorization on real-world data with the sparsity reported in Table 5, this is not plausible. Under misspecification, the bound in Lemma 9 gains an approximation error term involving inf_{f∈H} Σ(f-f_B)², which is neither bounded nor discussed. Therefore the formal claim that the trained MF model is NDCG/DCG-consistent is not established. The authors should either add a misspecification analysis or explicitly list this as a limitation of the theoretical results.","section":"Section 4.2 and Proposition 15"},{"comment":"The Taylor expansion at o0=0 is the only derivation of RG² and RG×, but its validity for the logit magnitudes reached by ALS is not established. Section 4.1 justifies the expansion by noting that the optimum is 'typically constrained close to the zero vector due to regularization,' yet the closed-form ALS updates in Eqs. (36)-(39) can produce logits far from zero, and Proposition 7's proof in Appendix A.3 only checks a Hessian bound under a 'single-click behavior' condition that is not formalized or verified on the trained models. Without a quantitative bound on the Taylor remainder along the optimization trajectory, the claim that RG losses are efficient approximations of softmax is not rigorously supported.","section":"Section 4.1 and Appendix A.3"},{"comment":"The convergence rate claim for ALS on the RG objectives is not proven for the specific losses in Eq. (31). Theorem 26 (Jain et al.) requires µ-marginal strong convexity, L-marginal strong smoothness, and M-robust bistability; the text asserts that these hold 'due to their quadratic structures' but does not verify them, and Eq. (31) is only marginally quadratic, not jointly. Robust bistability in particular is nontrivial for a quartic objective. The convergence-rate table is therefore a comparison of textbook rates rather than a guarantee for the proposed method. Please provide the verification or state the conditions under which the cited theorem applies.","section":"Section 4.5 and Appendix A.7"}],"minor_comments":[{"comment":"The text refers to 'RG 1' in the description of Figure 1, but RG1 is never defined; this should be RG² or RG×.","section":"Section 5.3"},{"comment":"The abstract and title say NDCG-consistency, but Definition 8 and the proofs in Section 4.2 concern DCG. Remark 22 discusses normalization, but the abstract and title should be aligned with the actual theorem statements.","section":"Abstract and Section 4.2"},{"comment":"The sentence 'the negative sampling numbers are 10, 10, 1 and 1, 100, 50 for all recommendation datasets' is garbled and should be restated with the intended per-dataset values.","section":"Appendix B.3"},{"comment":"The generalization bound derivation contains inconsistent constants: the proof replaces the factor (32eB)^d with (16eL)^d without comment, and the final displayed bound should be reconciled with the δ-solving step.","section":"Equation (33) and Appendix A.5"},{"comment":"The phrase 'a 3.3% average relative decrease compared with SM' is unclear; please specify the dataset and whether a decrease is a desirable outcome or not.","section":"Section 5.2, Findings 2"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper is within scope for a machine-learning or recommender-systems venue and the theoretical framework is promising. The decisive issue is the mismatch between the theory and the experiments: the authors should be asked to rerun the experiments with the exact derived weights or to restructure the claims. If the exact-weight version does not retain the reported gains, the contribution reduces to a theoretical analysis of a loss family that is not empirically competitive, which would significantly change the paper's value. The manuscript also cites its own prior work (Pu et al. 2024) appropriately; no disclosure concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely useful part of this paper is the derivation. Taking the Taylor expansion of softmax and sampled softmax at zero and rewriting the resulting quadratic forms as weighted squared losses is clean and insightful. The fact that the sampled-softmax weights reduce to something proportional to |I_x| and |I_x|(n+1)/N is a nice, concrete connection between sampling-based and non-sampling-based ranking losses. The Bregman-divergence framing of RG2 and RG× is also a good way to see why these squared losses inherit DCG-consistency from softmax. The generalization bound is standard but correctly assembled from existing pseudo-dimension and covering-number results. I would not call the consistency analysis groundbreaking, but it is careful and well worth having in the literature.\n\nThe soft spots are real and somewhat load-bearing. The experiments in Appendix B.2 explicitly replace the derived coefficients W and V with tuned hyperparameters α and β, so what is evaluated in Tables 3–4 and Figure 1 is not the exact RG2/RG× objective analyzed in Theorem 12. The paper itself warns that non-uniform weighting of positives and negatives can break the inverse-order-preserving property that the consistency proof relies on. If α and β are tuned separately, the trained model may be minimizing something whose DCG-consistency is not covered by the theory. If they are uniform constants, then the claim that the derived weights explain the performance is untested. Either way, the central empirical assertion is about a tuned member of the WRMF family, not the derived loss. That is a genuine gap. The theoretical sections are also premised on the Bayes-optimal scoring function being in the rank-64 hypothesis space (Section 4.2, after Eq. 25), which is almost certainly false for sparse real-world data. That weakens the practical force of the consistency theorem, though it is a standard kind of assumption in this literature.\n\nA few smaller things: the Taylor expansion at o0=0 is justified only by regularization and an appeal to small logits, which is plausible but not tested. The convergence-rate table summarizes existing results rather than proving anything new. There are also no standard deviations reported in the main tables, so the margins between RG2, RG×, and softmax—around 1–5% relative—could easily be noise.\n\nWho should read this? Anyone designing ranking losses for large-scale similarity search or recommendation. The derivation alone is worth the read. But the paper needs another round before acceptance: re-run the experiments with the derived W and V weights, or explicitly report which α and β values were chosen and show that the consistency-relevant weighting property is preserved, and add variance estimates. I would send it to a serious peer review, but I would ask for those changes before endorsing the empirical claims.","headline":"A solid theoretical bridge from softmax to weighted squared losses, but the experiments evaluate a tuned variant rather than the exact derived objective.","tokens_in":749,"tokens_out":2246,"would_cite":true,"duration_ms":35683,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Taylor expansion of the softmax loss at zero yields two quadratic ranking losses that inherit softmax's DCG-consistency, can be trained with closed-form alternating least squares updates, and match or beat softmax on three recommendation…","keywords":["similarity learning","ranking","softmax approximation","Bayes consistency","DCG consistency","alternating least squares","weighted squared loss","matrix factorization"],"falsifier":"Record the distribution of predicted scores during ALS training on any of the four datasets: if many scores move well beyond the region where the Taylor remainder is negligible (roughly $|o| > 1$), then the quadratic surrogate is no longer approximating softmax and $RG^\\times$ should visibly diverge from softmax's ranking. A second check is to train on synthetic data whose true relevance matrix has rank far above 64, where the DCG-consistency guarantee should fail.","tokens_in":25446,"feed_emoji":"⚡","tokens_out":15640,"duration_ms":135367,"temperature":0.7,"pith_summary":"This paper tries to show that the softmax ranking loss can be replaced by a quadratic surrogate without losing its ranking guarantees. Taylor-expanding the softmax and sampled-softmax losses at zero produces two new losses, the squared $RG^2$ and the interaction-preserving $RG^\\times$, which are non-sampling, admit closed-form alternating least squares updates, and are provably consistent with the DCG ranking metric (discounted cumulative gain) because they inherit the softmax's Bregman-divergence structure. Optimizing these losses with ALS on a matrix-factorization backbone reaches or beats softmax's MRR/NDCG on MovieLens, Amazon Electronics, and Steam in fewer epochs, while staying close on a link-prediction dataset. If the claims hold, large-scale ranking and similarity-learning systems can keep softmax-like ranking quality while replacing stochastic gradient training with faster, deterministic block updates, and the heuristic weights of weighted squared losses get a theoretical explanation.","feed_headline":"Quadratic losses match softmax ranking with far faster training","feed_subtitle":"RG² and RG× inherit softmax's ranking guarantees and reach its accuracy in a fraction of the epochs.","key_machinery":"The engine is the second-order Taylor expansion of the softmax loss at the all-zero score vector, which turns the nonlinear softmax into a quadratic form. The resulting loss is reorganized as a weighted squared loss, $RG^2 \\propto \\sum_{x,y} |I_x| (o^{(x)}_y + 1 - r_{x,y} N/|I_x|)^2$, with $RG^\\times$ subtracting the mean-interaction term $\\frac{1}{N}(1_N^T o^{(x)})^2$. Consistency is carried by the Bregman-divergence representation $L = D_\\phi(g(o^{(x)}), \\eta)$ with generating functions $\\phi_2(o) = |I_x|\\|o\\|^2$ and $\\phi_\\times(o) = |I_x| o^T (I - \\frac{1}{N}1_N1_N^T)o$ and inverse-order-preserving links $g_2(o) = 2|I_x|o$, $g_\\times(o) = 2|I_x|(I - \\frac{1}{N}1_N1_N^T)o$; this is what transfers softmax's DCG-consistency to the quadratic losses. Finally, the quadratic structure gives closed-form ALS updates in which each block update solves a small linear system, which is why training converges faster than SGD on softmax.","core_discovery":"The central claim is that the softmax loss can be approximated by a quadratic loss without sacrificing its ranking semantics. Expanding $L(o^{(x)}) = -\\log(\\mathrm{softmax}(o^{(x)})_y)$ at $o=0$ yields, up to constants, $-o^{(x)}_y + \\frac{1}{2N}\\|o^{(x)} + 1_N\\|^2 - \\frac{1}{2N^2}(1_N^T o^{(x)})^2$; keeping the first two terms gives the $RG^2$ loss and keeping the full quadratic gives $RG^\\times$. By rewriting these as weighted squared losses and identifying them as Bregman divergences with inverse-order-preserving link functions, the paper proves both are DCG-consistent, just like softmax, and shows $RG^2$ upper-bounds softmax under a single-click assumption. Because the losses are quadratic, a matrix-factorization instantiation can be optimized with alternating least squares, whose updates have closed forms and converge linearly rather than at SGD's sublinear rate. The empirical claim, on four datasets, is that $RG^2$ and $RG^\\times$ match or beat softmax ranking quality on three recommendation datasets and are only slightly behind on the link-prediction set while converging substantially faster.","pith_inferences":["The same Taylor-at-zero trick could be applied to other normalizing losses, such as sparsemax or hierarchical softmax, to obtain quadratic surrogates with their own consistency properties; the paper does not explore this.","Because the Taylor expansion is centered at zero and justified by regularization, the method implicitly predicts that strong $L^2$ regularization is part of the approximation: without it, RG losses should drift away from softmax behavior, which is a testable prediction.","The derived weight formula suggests a principled way to set weighted-squared-loss hyperparameters from a target negative-sampling budget rather than by heuristic tuning, which the experiments still do via $\\alpha$ and $\\beta$.","The generalization bound weakens as the embedding dimension $K$ grows, so on genuinely high-rank data the practical advantage of RG losses may come from the optimization speedup rather than from the DCG-consistency theorem."],"forward_implications":["Because $RG^2$ and $RG^\\times$ are quadratic, they can be optimized with ALS at per-iteration cost $O(T(|D|K^2 + (M+N)K^3))$, so softmax-style ranking can scale to very large item corpora without negative sampling.","Under the single-click assumption, $RG^2$ is an upper bound on softmax loss at their optima, so optimizing $RG^2$ also controls softmax behavior; this gives a principled explanation for why weighted squared losses such as WRMF work well for ranking.","The sampled-softmax-derived weights show that increasing the number of negative samples in sampled softmax is equivalent to adjusting positive and negative weights in a non-sampling squared loss, formally connecting the sampling and non-sampling paradigms.","The consistency and generalization results imply that, whenever the Bayes-optimal scoring function lies inside the rank-$K$ hypothesis space, optimizing $RG$ losses recovers the optimal DCG ranking rather than merely a proxy for it.","On the reported datasets, $RG^2$ and $RG^\\times$ match or exceed softmax on all recommendation benchmarks while the ALS optimization reaches a given ranking metric far earlier in wall-clock time."],"supporting_citations":[{"why":"Supplies the Bregman-divergence and inverse-order-preserving machinery used to prove softmax and the RG losses are DCG-consistent.","marker":"Ravikumar et al. (2011)"},{"why":"Establishes that softmax loss bounds log-NDCG, motivating the approximation of softmax as a ranking surrogate.","marker":"Bruch et al. (2019)"},{"why":"Provides the Top-k calibration and Bregman-divergence view of softmax that the consistency proof builds on.","marker":"Yang and Koyejo (2020)"},{"why":"Supplies the squared-loss consistency argument used as a stepping stone for proving RG2 consistency.","marker":"Cossock and Zhang (2006)"},{"why":"Gives the sampled-softmax bias correction whose form the Taylor expansion of SSM assumes.","marker":"Bengio and Senécal (2008)"},{"why":"Defines the weighted squared loss and ALS baseline (WRMF) that RG losses are derived from and compared against.","marker":"Hu et al. (2008)"},{"why":"Bounds the pseudo-dimension of low-rank matrix classes, which enters the generalization upper bound.","marker":"Srebro et al. (2004)"},{"why":"Provides the linear-convergence rate for ALS under marginal strong convexity used to justify the speedup.","marker":"Jain et al. (2017)"},{"why":"Provides the covering-number tool used to derive the generalization upper bound in Theorem 14.","marker":"Anthony et al. (1999)"}],"fun_headline_variants":["Quadratic losses match softmax ranking with faster training","Fast quadratic losses keep softmax's ranking edge","RG² and RG×: softmax accuracy at a fraction of the epochs","Softmax ranking quality, quadratic convergence speed","Quadratic surrogates match softmax, converge faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The formal guarantees presume both that the true relevance probabilities can be represented by the low-rank matrix-factorization model used (rank 64 in the experiments) and that the model's predicted scores stay close enough to zero for the Taylor expansion to hold.","fun_headline_variants_meta":{"raw":{"variants":["Quadratic losses match softmax ranking with faster training","Fast quadratic losses keep softmax's ranking edge","RG² and RG×: softmax accuracy at a fraction of the epochs","Softmax ranking quality, quadratic convergence speed","Quadratic surrogates match softmax, converge faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2711,"prompt_tokens":1070,"completion_tokens":1641,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":1562}},"tokens_in":686,"tokens_out":1641,"duration_ms":12551,"temperature":1.0,"reasoning_tokens":1562,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:48:25.707802+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the distribution of predicted scores during ALS training on any of the four datasets: if many scores move well beyond the region where the Taylor remainder is negligible (roughly $|o| > 1$), then the quadratic surrogate is no longer approximating softmax and $RG^\\times$ should visibly diverge from softmax's ranking. A second check is to train on synthetic data whose true relevance matrix has rank far above 64, where the DCG-consistency guarantee should fail.","supporting_citations":[{"cited_title":"On ndcg consistency of listwise ranking methods","cited_arxiv_id":null,"evidence_quote":"Supplies the Bregman-divergence and inverse-order-preserving machinery used to prove softmax and the RG losses are DCG-consistent."},{"cited_title":"An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance","cited_arxiv_id":null,"evidence_quote":"Establishes that softmax loss bounds log-NDCG, motivating the approximation of softmax as a ranking surrogate."},{"cited_title":"On the consistency of top-k surrogate losses","cited_arxiv_id":null,"evidence_quote":"Provides the Top-k calibration and Bregman-divergence view of softmax that the consistency proof builds on."},{"cited_title":"Subset ranking using regression","cited_arxiv_id":null,"evidence_quote":"Supplies the squared-loss consistency argument used as a stepping stone for proving RG2 consistency."},{"cited_title":"Adaptive importance sampling to accelerate training of a neural probabilistic language model","cited_arxiv_id":null,"evidence_quote":"Gives the sampled-softmax bias correction whose form the Taylor expansion of SSM assumes."},{"cited_title":"Generalization error bounds for collaborative prediction with low-rank matrices","cited_arxiv_id":null,"evidence_quote":"Bounds the pseudo-dimension of low-rank matrix classes, which enters the generalization upper bound."},{"cited_title":"Non-convex optimization for machine learning","cited_arxiv_id":null,"evidence_quote":"Provides the linear-convergence rate for ALS under marginal strong convexity used to justify the speedup."}],"review_version":1}