{"id":"00910a05-b628-4df7-b3f4-58dbd4ed9b3c","arxiv_id":"2411.12205","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SparseRec prunes recommendation embedding tables during training using NMF-based initialization, cumulative gradients for regrowth, and sparse backward passes, beating baselines on two datasets.","lead":"SparseRec is a new way to compress recommendation models for phones and small devices, by trimming the memory used for user and item profiles while training. It uses three tricks: better starting patterns for pruning, remembering useful pruned parts over time, and skipping heavy math in the backward pass.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Frequency-weighted sampling confounds the regrowth evaluation: inactive parameters can only be regrown from rows selected by Eq. 7–8, so the reported frequency–size correlations and cumulative-gradient gains are partly by construction, and no uniform-sampling or dense-gradient baseline is tested.","rationale":"The paper's central claim is that SparseRec achieves state-of-the-art accuracy while using sparse gradients and less memory. The frequency-weighted sampling is not an auxiliary detail: it is the mechanism that makes backward sparsity possible, and it directly determines which inactive parameters can be regrown. Because Eq. 7–8 restrict the candidate regrowth set to frequency-biased rows, the reported evidence for the method's own design choices (Figure 3 and the Table 2 comparison of cumulative vs instantaneous regrowth) cannot separate the effect of gradient accumulation from the effect of the sampling bias. This is a load-bearing gap: if a uniform-sampling baseline performs equally well, then the sparse-gradient contribution reduces to a memory trick without the claimed ability to identify important inactive parameters; if it performs substantially worse, the frequency bias is validated. The reader's weakest assumption identifies exactly this counterfactual, so I agree with the reader's assessment. Secondary issues such as missing error bars, hyperparameters selected from test-performance plots, and unreleased code further support keeping the verdict conditional, but they do not change the verdict category. The proposed uniform-sampling and dense-gradient-oracle test would directly resolve the main uncertainty, and the verdict should remain CONDITIONAL until that evidence appears.","tokens_in":16122,"tokens_out":9519,"duration_ms":110454,"concrete_test":"Run the full SparseRec pipeline with the sampling probabilities in Eq. 8 replaced by a uniform distribution over user and item rows (p_i = 1/(m+n)), keeping NMF initialization, cumulative gradients, DeltaT, all hyperparameters, and seeds fixed, over the 18 settings in Table 1. Report mean and standard deviation of R@20/N@20 over at least 5 seeds, plus a dense-gradient oracle (omega=1) as an upper bound. If uniform sampling matches or beats frequency-weighted sampling within seed noise, the frequency-weighting contribution collapses; if frequency-weighted sampling is significantly better, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"At the core of SparseRec is the claim that the sparse-gradient/regrowth scheme (Section 4.2–4.3) finds inactive parameters that matter. But Eq. 7–8 sample the gradient subset S from a Softmax of user/item frequencies, Eq. 11 accumulates gradients only over S, and Eq. 12 regrows from the top-k of that accumulated C. Thus a pruned parameter can be regrown only if its row was selected in S, and high-frequency rows dominate selection. The frequency–embedding-size correlations in Figure 3 and the cumulative-vs-instantaneous gain in Table 2 are therefore partly built into the mechanism rather than evidence that frequency-weighted sampling is the right inductive bias. The ablations in Table 2 vary mask initialization (NMF vs uniform) and gradient accumulation (cumulative vs instantaneous), both while keeping the same frequency-weighted S; they never compare S drawn uniformly at random, nor a dense-gradient oracle (omega=1). If truly valuable inactive parameters are concentrated in the long tail, this sampling will systematically miss them, and the 16/18 SOTA results could be carried by frequent users/items only. The paper also does not report the sampling ratio omega used in the main Table 1, making it impossible to verify the memory claim (Eq. 15) for the actual configuration.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SparseRec, a dynamic sparse training (DST) method for embedding tables in GNN-based recommender systems targeting on-device deployment. It makes three contributions: (1) initializing the binary mask matrix via Nonnegative Matrix Factorization (NMF) on the user-item interaction matrix; (2) regrowing pruned parameters according to cumulative gradients accumulated over several training steps rather than instantaneous gradients; and (3) computing gradients only for active parameters plus a frequency-weighted random sample of embedding vectors, thereby keeping the backward pass sparse. The method is evaluated on Gowalla and Yelp with XSimGCL, MGDCF, and LightGCN as base recommenders at three density ratios, reporting top scores in 16 of 18 settings and up to 11.79% relative improvement over the second-best baseline, with a theoretical memory-complexity advantage over prior DST methods.","tokens_in":16407,"tokens_out":12823,"duration_ms":126897,"significance":"SparseRec addresses a practically important problem—memory-efficient, on-device recommendation—and the experimental evaluation is reasonably broad: two datasets, three base recommenders, three density ratios, and several established baselines. If the method is sound as described, the NMF initialization and cumulative-gradient regrowth would be useful additions to the DST literature for recommender systems. The paper also includes a careful memory-complexity analysis (Section 4.4) that separates it from earlier dense-gradient DST methods. However, the central sparse-gradient mechanism needs a crucial clarification about how gradients for masked-out (inactive) parameters are obtained, and several experimental details (the sampling ratio used in the main results, run-level statistics) are missing. These issues do not render the empirical claims impossible, but they must be addressed before the method can be assessed as reproducible and fully supported.","major_comments":[{"comment":"This is a load-bearing correctness concern because the sparse-gradient and regrowth mechanisms are the core contributions. If the gradients for inactive parameters are indeed zero, the reported performance improvements must be attributed to some other, unspecified mechanism.","section":"Section 4.2, Eq. (9)–(11); Algorithm 1"},{"comment":"The memory advantage is a central selling point of the paper, so the missing ω makes the main memory claim unverifiable.","section":"Section 5.5.2 and Table 1"},{"comment":"Given that the paper claims superiority in 16/18 settings, this statistical evidence is load-bearing for the empirical contribution.","section":"Section 5.3"},{"comment":"This ablation is important because the sparse-gradient mechanism is a claimed contribution; without it, the reader cannot tell whether the frequency-based sampling is justified or merely an arbitrary choice that happens not to hurt performance.","section":"Table 2 and Section 4.2, Eqs. (7)–(8)"}],"minor_comments":[{"comment":"Typos: 'they typically initializes the mask matrix' should be 'they typically initialize the mask matrix'; 'an lightweight embedding method' should be 'a lightweight embedding method'.","section":"Abstract and Introduction"},{"comment":"The heading 'Hyerparameter' appears twice and should be 'Hyperparameter'.","section":"Section 5.5"},{"comment":"The notation 'p^U / p^U' appears to be a typo; it should likely be 'p^U / p^V' for the item sampling distribution.","section":"Eq. (7)"},{"comment":"The y-axis label 'User emb. size' appears on the item-frequency panels; it should read 'Item emb. size'.","section":"Figure 3"},{"comment":"The claim that 'an ω close to zero essentially results in random regrowth' is imprecise: with tiny ω, only a very small candidate set is available, but the sampling is still frequency-weighted, not uniform random.","section":"Section 5.5.2"},{"comment":"The regrowth selection should explicitly exclude active parameters from the top-k over |C|, since G is required to be disjoint from A (as shown in Figure 2). The text should state that C is zeroed or filtered for active parameters before applying top-k.","section":"Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty statement and contribution list are clear, but the authors may wish to check whether the most recent related work on dynamic sparse training for recommendation (beyond DSL) is fully covered. Releasing code would substantially help reproducibility, especially given the ambiguity in the gradient computation for inactive parameters."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the substance: SparseRec combines three established ideas—NMF-based mask initialization, cumulative-gradient regrowth, and frequency-sampled sparse gradients—into a DST pipeline for GNN-based recommenders. The combination is new, and the experiments are reasonably broad: two datasets, three base recommenders, three density ratios. The 16/18 win rate is real on the face of it, and the memory analysis in Eq. 15 is a useful back-of-the-envelope comparison. The NMF initialization, while simple, is a sensible alternative to random masks and seems to help in the ablation (Table 2).\n\nThe soft spots are real but not fatal. The biggest one: the regrowth evaluation never tests a uniform-sampling baseline. Since Eq. 7–8 sample S by frequency, and Eq. 12 can only regrow rows in S, the frequency–size correlation in Figure 3 is partly by construction. The paper tries to address this by setting omega=1 in Section 5.5 and showing the correlation persists; that is a partial answer, but it is not the same as comparing performance against a uniform-sampling ablation in the actual pipeline. Without that control, we cannot tell whether the gains over DSL come from the frequency prior or from the cumulative-gradient mechanism. Second, the paper never reports the omega value used in Table 1, so the memory claim cannot be checked for the actual configuration. Third, the hyperparameter plots (Figures 4–5) do not clearly state whether they report validation or test metrics; if the latter, the hyperparameter choices are leaked. Fourth, the t-test claim is unverifiable without standard deviations or the number of runs.\n\nOn balance, this is a solid empirical contribution to the on-device recommendation subfield. It is not a breakthrough, and the improvements are mostly small (the headline 11.79% is one of 18 settings), but the method is clearly described, the baselines are appropriate, and the problem is well-motivated. The right call is to send it to peer review with the expectation of a revision that adds the missing controls and reporting. I would specifically demand the uniform-sampling ablation and the omega value for the main configuration.","headline":"A credible empirical DST method for on-device collaborative filtering with a few evaluation gaps, most notably a missing uniform-sampling control and an unspecified omega in the main results.","tokens_in":16927,"tokens_out":3978,"would_cite":true,"duration_ms":41752,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that dynamic sparse training for recommendation embeddings can be made fully sparse in both forward and backward passes, and that the resulting method reaches state-of-the-art Recall@20 and NDCG@20 in 16 of 18 settings on…","keywords":["Recommender Systems","Model Pruning","Collaborative Filtering","Dynamic Sparse Training","Embedding Tables","Nonnegative Matrix Factorization","Sparse Gradients","On-Device Recommendation"],"falsifier":"Run SparseRec on a long-tail-heavy dataset with a dense-gradient oracle that keeps the NMF mask and cumulative regrowth but computes all gradients, and compare Recall@20 at the same density; a large gap would show the sparse-gradient sampling is the bottleneck, while swapping the frequency-based sampler for uniform row sampling would isolate whether frequency weighting or plain stochastic exploration drives the gains.","tokens_in":15944,"feed_emoji":"📱","tokens_out":10906,"duration_ms":100217,"temperature":0.7,"pith_summary":"Recommender systems on phones and other memory-limited devices are dominated by the embedding table, which stores a dense vector for every user and item. The paper argues that dynamic sparse training, which prunes most of that table, has three correctable weaknesses: the initial mask is random, regrowth decisions come from a single batch, and backpropagation still computes dense gradients. SparseRec replaces each piece: a nonnegative matrix factorization of the interaction matrix supplies the initial mask, cumulative gradients drive regrowth, and frequency-based sampling decides which vectors get gradients, so sparsity holds in both forward and backward passes. If these claims hold, a recommender can be trained on-device at a fraction of the memory cost while matching or beating heavier baselines. On two public datasets and three base models, the paper reports the best metric in 16 of 18 settings and up to 11.79% relative gain over the second-best baseline.","feed_headline":"SparseRec tops 16 of 18 sparse-embedding settings","feed_subtitle":"Sparse matrix masks, cumulative gradients, and frequency-sampled updates preserve top-20 accuracy at lower memory","key_machinery":"The argument rides on three pieces of machinery. The first is a binary mask built by binarizing the nonnegative matrix factorization (NMF) of the interaction matrix, which supplies a naturally sparse, data-driven initialization instead of a random uniform mask. The second is a cumulative gradient table that sums sparse gradients over the exploration interval and selects regrown parameters by top-k magnitude, so reactivation reflects behavior over many batches. The third is multinomial sampling over user and item frequencies, which chooses which embedding rows receive gradient computation at all, leaving the backward pass sparse. A short memory analysis ties these together: with density $d$ and sampling ratio $\\omega$, peak training storage is at most $(2d+2\\omega)s(m+n)$ parameters, which is below the $(d+1)s(m+n)$ cost of conventional DST exactly when $\\omega \\le (1-d)/2$. That inequality is what guarantees the method uses less memory than dense-gradient DST.","core_discovery":"The central claim is that a dynamic sparse training pipeline for graph-based recommenders can be made genuinely sparse in both directions of training without sacrificing accuracy, provided three mechanisms are changed together. SparseRec initializes the binary pruning mask by binarizing a nonnegative matrix factorization of the user-item interaction matrix, giving the sparse structure a data-driven starting point rather than a random one. It regrows inactive parameters according to gradients accumulated over the exploration window, so reactivated parameters serve users and items that matter across many batches rather than only the current one. It computes gradients only for active parameters plus a frequency-sampled subset of embedding rows, which keeps the backward pass sparse. The paper reports that this combination reduces peak memory relative to dense-gradient DST whenever the sampling ratio satisfies $\\omega \\le (1-d)/2$, and delivers the best metric in 16 of the 18 evaluated settings.","pith_inferences":["A uniform-sampling ablation would separate the contribution of frequency weighting from plain stochastic exploration; the paper's ablations vary initialization and gradient accumulation but not the sampling distribution.","Because cumulative gradients push capacity toward frequent users and items, SparseRec could be combined with variable-size embedding methods that let the model decide where to spend dimensions, rather than fixing all rows to the same width.","The same recipe of data-driven mask, accumulated-gradient regrowth, and row-sampled sparse backward passes may generalize to other embedding-heavy models such as sequential or categorical-feature recommenders, although the paper only evaluates graph-based collaborative filtering.","In applications where rare users or items carry the most information, the frequency-based sampler is a liability; a salience-based sampler would be a natural variant the paper does not explore."],"forward_implications":["Sparse training can run end to end on-device: the backward pass no longer allocates a dense gradient table, because gradients are computed only for active parameters plus the frequency-sampled rows.","Peak training memory is bounded by $(2d+2\\omega)s(m+n)$ parameters, which is less than the $(d+1)s(m+n)$ cost of conventional DST whenever the sampling ratio satisfies $\\omega \\le (1-d)/2$.","NMF-based mask initialization outperforms random uniform initialization in the paper's ablations under both cumulative and instantaneous regrowth, so the starting structure of the sparse network matters.","Cumulative-gradient regrowth assigns larger effective embeddings to frequent users and items, with Pearson correlations around 0.8, while instantaneous gradients produce a weaker frequency-size alignment.","The method's largest relative improvement over the best baseline appears at the lowest density ratio (6.25%), suggesting the sparsest regimes benefit most from targeted regrowth."],"supporting_citations":[{"why":"Provides the dynamic sparse training formulation from scratch that the method extends to embeddings.","marker":"[2]"},{"why":"Establishes the suboptimality of random uniform sparse initialization that motivates the NMF mask.","marker":"[3]"},{"why":"DSL is the main dense-gradient DST baseline whose memory cost $(d+1)s(m+n)$ the paper's bound is designed to beat.","marker":"[44]"},{"why":"PEP supplies the learnable-threshold embedding pruning baseline and the $3s(m+n)$ memory term in the comparison.","marker":"[23]"},{"why":"Shows that frequent features need larger embedding sizes, which motivates frequency-based sampling of rows for gradients.","marker":"[22]"},{"why":"Inspires the sparse-gradient idea of sampling inactive parameters, but SparseRec replaces its per-parameter probabilities with frequency-based row sampling.","marker":"[9]"},{"why":"Supplies one base graph-based recommender used in all evaluation settings.","marker":"[6]"},{"why":"Supplies the other base graph-based recommender used in the main tables and ablations.","marker":"[10]"}],"fun_headline_variants":["SparseRec: sparse training both ways, wins 16 of 18","SparseRec uses NMF masks and cumulative gradients for sparse Rec","SparseRec: no dense gradients, better memory, top results","SparseRec: data-driven mask initialization boosts sparse training","SparseRec: sparse forward and back, 16 of 18 best"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a frequency-weighted random sample of embedding rows reveals which inactive parameters are most valuable to reactivate; if the important dormant parameters live in rows belonging to rare users or items, the sampling can miss them and regrowth slides toward random selection.","fun_headline_variants_meta":{"raw":{"variants":["SparseRec: sparse training both ways, wins 16 of 18","SparseRec uses NMF masks and cumulative gradients for sparse Rec","SparseRec: no dense gradients, better memory, top results","SparseRec: data-driven mask initialization boosts sparse training","SparseRec: sparse forward and back, 16 of 18 best"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000345,"raw_usage":{"total_tokens":1922,"prompt_tokens":1002,"completion_tokens":920,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":827}},"tokens_in":618,"tokens_out":920,"duration_ms":9863,"temperature":1.0,"reasoning_tokens":827,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:48:38.967579+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SparseRec on a long-tail-heavy dataset with a dense-gradient oracle that keeps the NMF mask and cumulative regrowth but computes all gradients, and compare Recall@20 at the same density; a large gap would show the sparse-gradient sampling is the bottleneck, while swapping the frequency-based sampler for uniform row sampling would isolate whether frequency weighting or plain stochastic exploration drives the gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DSL is the main dense-gradient DST baseline whose memory cost $(d+1)s(m+n)$ the paper's bound is designed to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PEP supplies the learnable-threshold embedding pruning baseline and the $3s(m+n)$ memory term in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that frequent features need larger embedding sizes, which motivates frequency-based sampling of rows for gradients."},{"cited_title":"Always-Sparse Training by Growing Connections with Guided Stochastic Exploration","cited_arxiv_id":"2401.06898","evidence_quote":"Inspires the sparse-gradient idea of sampling inactive parameters, but SparseRec replaces its per-parameter probabilities with frequency-based row sampling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the other base graph-based recommender used in the main tables and ablations."}],"review_version":1}