{"id":"19247bd7-17d1-4cb6-aebd-2ef43a93c83b","arxiv_id":"2411.13052","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Shaver prunes CTR model embedding tables to arbitrary sparsity in a single shot using Shapley-value attribution and a field-aware codebook, with accuracy close to retrained baselines.","lead":"Shaver is a method for shrinking the embedding tables of content-based recommendation models in one shot, without retraining for each target size. It estimates how much each embedding parameter matters using Shapley values and fills in pruned parameters with field-specific placeholder values.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"One-hot fields are load-bearing for Theorem 4.3: with ≥2 active features per field, the corresponding-player map collapses, Algorithm 1 line 9 is underdetermined, and the O(md|D|) complexity claim fails. All three datasets are single-valued, so multi-hot is never tested.","rationale":"I read the paper in good faith and independently verified the load-bearing mathematics before critiquing. Theorem 4.3's proof (Appendix A) is algebraically correct: the identity in Eq. (21) checks out and Eq. (22) is a valid Chu–Vandermonde summation, so under one-hot per field the equality of the parameter-level and field-level Shapley values, and the O(md|D|) complexity reduction, genuinely hold. Algorithm 1 is a standard unbiased permutation-sampling estimator for that quantity, and Appendix B.3's exact-vs-estimated toy verification (mean abs error 0.0021) supports the estimator; the on-device benchmark (Table 8) also concretely supports the deployment story. The single most load-bearing assumption is the one-hot field structure of Eq. (1). It is indispensable to Definition 4.2's unique correspondence, to the null-player argument, and to the subset-count in the proof. Multi-hot fields are a normal part of content-based recommender inputs (a movie has several genres; a user has several interest tags), and under multi-hot the correspondence becomes one-to-many, the field-level marginal becomes a joint effect, Algorithm 1's line 9 loses its unique target, and the computed scores cease to be the Definition 4.1 Shapley values. Valid per-parameter attribution then costs O(nd|D|) again. Because all three benchmarks are preprocessed to single-valued fields (Appendix C.2), no experiment breaches the assumption, so the reported numbers stand — but the generality of the headline claims does not, and the paper never flags this boundary. This is a correctness/scope risk of the paper's presentation, not an internal inconsistency. Secondary observations, none of which I treat as load-bearing: (i) the codebook is optimized under a uniformly random mask (Eq. 14) while deployed with a Shapley-ranked mask, and the Shapley scores are computed under zero-out removal while deployment imputes codebook values — a justification gap for the codebook's claimed benefit, though the experiments show it helps; (ii) per-parameter Shapley estimates for rare features average only over instances where that feature is active, so their noise is larger than the field-level O(1/|D|) analysis of Appendix B.2 suggests; (iii) the paper's own RQ3 results on KDD show finetuning can degrade AUC (0.7870→0.7754 at t=50%), which the authors attribute to validation leakage — an acknowledged, off-path limitation; (iv) no error bars, so several gaps in Table 2 are within the 0.001-AUC significance convention the paper cites. The reader's CONDITIONAL verdict remains the right one: the method is sound within its stated one-hot domain and its experiments are consistent, but the paper should disclose the one-hot scope and either extend Theorem 4.3 or adapt Algorithm 1 for multi-hot fields before the CRS-generality claims stand. My read does not move the verdict.","tokens_in":23646,"tokens_out":34493,"duration_ms":312979,"concrete_test":"Build a small multi-hot CTR task (e.g., DeepFM on MovieLens-1M with multi-label genres as one field, plus a synthetic set with n ≤ 40 features, d = 8, and a field with 2–3 active features per instance). Arm 1: for ~50 instances, compute exact per-parameter local Shapley values φ_v^{x,y}(i,c) by full subset enumeration (Eq. 5) and compare with field-game values φ_u^{x,y}(j,c) from Definition 4.2 and Eq. 12. If φ_v ≠ φ_u for any instance with ≥2 active features in a field, or if Algorithm 1 line 9 has no unique (i,c), the theorem's boundary is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headlined efficiency gain — O(md|D|) vs O(nd|D|) (Theorem 4.3) — is valid only if each feature field activates exactly one feature per instance (Eq. 1). That single-activation condition is what makes the 'corresponding player' of Definition 4.2 unique, what makes all non-activated parameters null players, and what makes the Appendix A subset-counting valid: both the map g(S_v)→S_u and the binomial count C(nd−md, |S_v|−|S_u|) (Eqs. 19–22) presuppose exactly one activated parameter per field, per dimension. I verified the proof's algebra (Eq. 21 is a correct identity; Eq. 22 is a valid Chu–Vandermonde sum), so the theorem is internally sound for its stated domain. The domain is the problem. Multi-hot fields — movie genres, user interest tags, item attributes — are routine in content-based recommenders. With two or more active features in a field, the correspondence is one-to-many: (i₁,c), (i₂,c), ... all correspond to (j,c). The field-level marginal Δ(j,c|S_u) is then the joint effect of removing every active feature's c-th coordinate, which equals no single parameter's marginal; the counting over nd−md free parameters breaks; and Algorithm 1 line 9, 'Find (i,c) as (j,c)'s corresponding player', is ambiguous about which active feature receives each attribution. The output is no longer the per-parameter Shapley value of Definition 4.1, and recovering true per-parameter Shapley values reverts to the O(nd|D|) procedure of Section 4.2. Criteo, Avazu, and KDD are all preprocessed to single-valued fields (Appendix C.2), so no reported experiment exercises the multi-hot regime. The paper presents one-hot as the general CTR formulation (Eq. 1) and does not disclose that Theorem 4.3, Algorithm 1, and the O(md|D|) claim are all conditional on it — a scope gap, not an internal contradiction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Shaver, a single-shot embedding-pruning method for content-based recommender systems (CTR models). Shaver assigns each embedding parameter a Shapley value as an attribution score, so that a trained model can be pruned to any parameter budget without retraining. To make Shapley computation tractable, the paper proves Theorem 4.3, which reduces the complexity from O(nd|D|) to O(md|D|) under the assumption that each feature field has exactly one activated feature per instance. The paper also proposes a field-aware codebook to replace zero-padding for pruned parameters, with a closed-form solution. Experiments on Criteo, Avazu, and KDD with DeepFM and DCN-Mix backbones compare Shaver with non-single-shot and single-shot baselines, and an on-device benchmark is reported.","tokens_in":24029,"tokens_out":4199,"duration_ms":42727,"significance":"If the theoretical reduction and the empirical claims hold, Shaver would be a useful contribution: it targets a practical single-shot pruning setting, provides a nontrivial complexity reduction, ships code, and includes a synthetic-data verification that the estimated Shapley values are close to exact ones. The experiments cover three large-scale datasets, two backbones, many sparsity levels, and an edge-device deployment benchmark, which is a real strength. However, the central efficiency theorem depends on a single-activation-per-field assumption that is not stated as a limitation, and the experimental sections report no variance across seeds. Both issues bound the significance of the work as written.","major_comments":[{"comment":"Theorem 4.3 and its proof are internally sound only when every feature field activates exactly one feature per instance, as encoded in Eq. (1). Under multi-hot fields—e.g., movie genres or user interest tags—the 'corresponding player' of Definition 4.2 is not unique, the map g in Eq. (16) is no longer single-valued, Algorithm 1 line 9 is ambiguous, and the counting argument behind the O(md|D|) complexity claim collapses. Multi-hot categorical features are routine in content-based recommendation, so this is a load-bearing restriction on the paper's central claim. The authors should either state this assumption as a clear scope limitation or extend the method and theorem to multi-hot fields (for instance, by decomposing fields into per-feature subgames).","section":"§4.3, Definition 4.2, Appendix A"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported anywhere. Many of the claimed advantages are based on AUC differences of 0.0001–0.001, which are within the range where seed-to-seed variation matters. Since the headline contribution is 'competitive performance' across budgets, single-run results are not sufficient to support the comparison. Please report means and standard deviations over at least three seeds, or explicitly state that only one seed was used and discuss the implications.","section":"§5.2–5.4, Tables 2 and 3, Figure 2"},{"comment":"Equation (14) defines the optimal codebook with respect to a uniformly sampled pruned set Q of size B, but in the actual pruning procedure Q is the set of parameters with the lowest Shapley values. Consequently, the closed-form solution in Eq. (15) is optimal for a different distribution over pruning masks than the one actually used. The paper calls C* 'optimal' and attributes performance gains to it, so this mismatch is a gap between the derivation and the deployed algorithm. Either derive the codebook for the actual selection rule or explicitly present it as a heuristic approximation.","section":"§4.4, Eq. (14), Appendix A"}],"minor_comments":[{"comment":"The notation R(x,y) is used both for a random permutation and as a paired sample; this is confusing because R was previously an ordered set. Please rename one of them.","section":"§4.2, Eq. (10)"},{"comment":"There is a typo: 'with a quality, decent-sizedD' should read 'with a quality, decent-sized D'.","section":"§4.2, text after Eq. (10)"},{"comment":"The word 'Shapely' in 'efficient Shapely value computation' is misspelled; it should be 'Shapley'.","section":"§4.3, first paragraph"},{"comment":"The dataset preprocessing description does not state whether any field in Criteo, Avazu, or KDD is multi-hot. Since the theorem assumes single activation per field, please make explicit whether the preprocessed datasets satisfy this condition.","section":"Table 1 and Appendix C.2"},{"comment":"The x-axis label 'Frequency Bins' is not self-explanatory; please clarify whether bins are equal-frequency or equal-width, and whether the three bins contain equal numbers of features.","section":"Figure 4"},{"comment":"The phrase 'can cause data leakage' is inaccurate if 'leakage' refers to test-set contamination: Shapley values are computed from training and validation data, and the test set is held out. Please rephrase to describe the selection effect on fine-tuning more precisely.","section":"§5.4, closing paragraph"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the unstated single-activation assumption in Theorem 4.3. If the authors can either extend the method to multi-hot fields or clearly restrict the claims, and if they add variance reporting, the paper would be publishable. The codebook derivation mismatch is also worth fixing, but it is less severe because the empirical results could still hold. There is no indication of citation or novelty issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a real method paper, not a hype job. The new piece is Theorem 4.3, which collapses Shapley estimation over n*d embedding parameters to m*d field-level players for the per-instance game, cutting the cost from O(nd|D|) to O(md|D|). I checked the appendix proof line by line; the subset-counting and the Chu–Vandermonde step are correct. The toy experiment in B.3, comparing exact and estimated Shapley values, supports the unbiasedness claim. The field-aware codebook closed form (Eq. 15) is simple, cheap, and the experiments suggest it helps at high sparsity. Three large datasets, two backbones, and a wide range of sparsity budgets is solid, and Shaver-Codebook is usually best-or-second-best even against baselines that retrain per budget. The authors are also honest where things go wrong: fine-tuning sometimes hurts, and codebook can underperform zero-padding on Avazu.\n\nThe soft spots, in rough order of importance. First, the one-hot assumption is load-bearing and understated. Eq. (1) defines the input as a concatenation of one-hot field encodings, and Definition 4.2's 'corresponding player' is unique only because exactly one feature is active per field. The proof's map g and the binomial count over n*d - m*d free parameters break when a field can have two or more active features; Algorithm 1 line 9 then has no unique answer, and the O(md|D|) claim is not justified. All three datasets are preprocessed to single-valued fields, so the multi-hot regime is never tested. This is a scope limitation, not an internal contradiction, but it needs to be stated and ideally addressed (e.g., approximate per-feature attribution within a field). Second, there are no error bars or multiple seeds anywhere. The field's convention treats 0.001 AUC as significant, but with no variance we can't tell whether the reported gaps are noise. Third, the Shapley values are computed with zero-out removal, yet final pruning uses the codebook; the attribution scores are never recomputed for the codebook imputation. The codebook helps empirically, but this mismatch deserves discussion. Fourth, the runtime claim for Shapley computation (2.5 hours on Criteo) lacks hardware and baseline context – minor.\n\nWho's this for: anyone working on embedding compression, on-device recommendation, or Shapley-based pruning. The central result holds for the domain it actually tests, and the limitations are fixable in a revision. I would send it to serious referees. For me, the paper deserves a chance; the main thing I'd ask the authors to do is make the single-activation constraint explicit and either test multi-hot data or clearly delimit the claim.","headline":"Shaver is a credible single-shot embedding pruner for CTR models, with a theorem that genuinely buys a complexity reduction; just know the theorem and the algorithm only work when every field activates exactly one feature per row, and the paper doesn't say that loudly.","tokens_in":24646,"tokens_out":3379,"would_cite":true,"duration_ms":32547,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Shaver is a single-shot embedding pruning method that gives every embedding parameter a Shapley value as a fair attribution score, so one well-trained recommender model can be compressed to any parameter budget without retraining.","keywords":["embedding pruning","Shapley value","single-shot pruning","content-based recommendation","click-through rate prediction","on-device recommender systems","field-aware codebook","cooperative game theory"],"falsifier":"Take a small CTR model with a field whose instances can activate two features simultaneously, enumerate all parameter subsets to compute exact Shapley values, and compare them with Shaver's field-level estimates on the same data; any material gap in the attribution of an active parameter would falsify Theorem 4.3's equality. A simpler check is whether the method's validation AUC degrades sharply when a multi-hot field is included in an otherwise one-hot dataset.","tokens_in":23457,"feed_emoji":"✂️","tokens_out":6284,"duration_ms":90829,"temperature":0.7,"pith_summary":"Shaver is a method for compressing the embedding tables of content-based recommender systems in a single shot, without retraining for each target size. The paper's central claim is that every embedding parameter can be given a Shapley value as a fair attribution score, and that pruning the lowest-scoring parameters to any budget preserves recommendation accuracy well. A theorem about corresponding players is what makes this tractable: because each feature field contributes exactly one active feature per instance, the parameter-level Shapley game collapses to a field-level game, cutting estimation cost from $O(nd|D|)$ to $O(md|D|)$. A field-aware codebook fills pruned entries with learned per-field placeholder values instead of zeros, limiting the damage to feature-interaction computations. If correct, Shaver lets a single well-trained model be adapted to heterogeneous device capacities, federated settings, or changing memory and battery conditions.","feed_headline":"One scoring pass prunes recommender embeddings to any size","feed_subtitle":"Fair-attribution scores pick which embeddings to delete; one trained model adapts to any device budget with no retraining.","key_machinery":"The load-bearing mechanism is the corresponding-player identity (Definition 4.2 and Theorem 4.3): for a data instance in which feature $i$ of field $j$ is the sole activated feature, the Shapley value of embedding parameter $(i,c)$ under the parameter-level value function equals the Shapley value of field-slot player $(j,c)$ under the field-level value function. This identity is what collapses the combinatorial game over $nd$ parameters to one over $md$ field slots, making the Monte Carlo estimation in Algorithm 1 tractable. Two auxiliary mechanisms support the result: the local-value formulation of Eq. (10), which turns global Shapley values into averages over per-instance permutations, and the field-aware codebook of Eqs. (14)-(15), which imputes pruned parameters with per-field weighted-average embeddings rather than zeros.","core_discovery":"On the paper's own terms, the central discovery is that Shapley-value attribution, previously considered too expensive for parameter-level pruning in recommender systems, can be computed efficiently for embedding tables because of the one-hot structure of categorical CTR data. Definition 4.2 pairs each activated embedding parameter $(i,c)$ with a field-level player $(j,c)$, and Theorem 4.3 proves their Shapley values are equal for that instance. Consequently, estimating attribution over the $n$ features reduces to estimating it over the $m$ fields, and Algorithm 1 accumulates unbiased Monte Carlo estimates in $O(md|D|)$ forward passes. The pruned model replaces low-scoring entries with per-field codebook values that solve in closed form, giving a single-shot compressor that the experiments show matches or beats trained-from-scratch baselines on Criteo, Avazu, and KDD across sparsity rates from 20% to 99.9%.","pith_inferences":["Our inference: the one-hot assumption in Eq. (1) is the hinge. If a deployment uses multi-hot fields, user-defined dense features, or feature crosses, the corresponding-player equality no longer holds exactly; a fallback would be to split multi-hot fields into per-feature singleton games or treat the active set as a coalition, at extra cost.","Our inference: the codebook's tendency to spend less budget on high-frequency features (Figure 4) suggests a principled pruning policy could be derived from feature frequency and field interactions, not just raw attribution magnitude.","Our inference: Shaver's attribution scores are computed from a trained model, so the method should transfer to other interaction-heavy architectures such as factorization machines and cross networks; a natural test is whether attribution scores computed on DeepFM also prune a DCN-v2 model effectively.","Our inference: because estimation only needs forward passes, the same scoring procedure could be applied periodically in a streaming setting to adapt an embedding table to drift, something the paper motivates but does not evaluate."],"forward_implications":["A trained CTR model can be shipped once and specialized to any device memory budget by ranking parameters and truncating, with no per-budget retraining or mask search.","The $O(md|D|)$ cost means attribution can be recomputed on-device or in streaming and federated settings whenever the model or data distribution shifts.","Because the codebook stores only $md$ values, replacing zero-padding adds negligible storage while preserving more of the dot-product and element-wise feature interactions that CTR models rely on.","The method extends beyond the three reported datasets to any categorical CTR model whose input is a concatenation of one-hot field encodings, including DeepFM and DCN-v2 backbones.","Combining Shaver's scores with post-pruning fine-tuning yields further gains in several settings, though the paper notes occasional degradation from validation-set leakage."],"supporting_citations":[{"why":"Defines Shapley value, the attribution principle Shaver assigns to every embedding parameter.","marker":"[50]"},{"why":"Supplies the local-value Monte Carlo formulation that turns global Shapley values into per-instance permutation averages.","marker":"[12]"},{"why":"Provides the polynomial Monte Carlo approximation scheme Shaver's estimator is based on.","marker":"[5]"},{"why":"Argues Shapley values satisfy null-player, symmetry, and efficiency properties desirable for pruning, motivating their use.","marker":"[2]"},{"why":"Introduces single-shot pruning by connection sensitivity, the prior setting Shaver contrasts with and extends.","marker":"[28]"},{"why":"Presents single-shot embedding dimension search, a baseline and comparison point for Shaver's single-shot claim.","marker":"[44]"},{"why":"Documents the importance of placeholder values in Shapley estimation, supporting the field-aware codebook design.","marker":"[6]"},{"why":"Defines the DeepFM backbone and second-order feature interactions that motivate non-zero placeholder values.","marker":"[18]"},{"why":"Defines the DCN-v2 and DCN-Mix backbones used in the main experiments.","marker":"[57]"}],"fun_headline_variants":["Shapley scores make embedding pruning a one-shot deal","Fair attribution prunes recommender embeddings in one pass","Game theory strips embedding tables to any size, no retraining","One-shot embedding pruning via Shapley values for recommender systems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's efficiency rests on a single one-hot premise: every field activates exactly one feature per instance, so an inactive embedding parameter is a null player and the field-level game exactly mirrors the parameter-level game.","fun_headline_variants_meta":{"raw":{"variants":["Shapley scores make embedding pruning a one-shot deal","Fair attribution prunes recommender embeddings in one pass","Game theory strips embedding tables to any size, no retraining","One-shot embedding pruning via Shapley values for recommender systems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1632,"prompt_tokens":1000,"completion_tokens":632,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":563}},"tokens_in":616,"tokens_out":632,"duration_ms":11913,"temperature":1.0,"reasoning_tokens":563,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:53:27.308869+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small CTR model with a field whose instances can activate two features simultaneously, enumerate all parameter subsets to compute exact Shapley values, and compare them with Shaver's field-level estimates on the same data; any material gap in the attribution of an active parameter would falsify Theorem 4.3's equality. A simpler check is whether the method's validation AUC degrades sharply when a multi-hot field is included in an otherwise one-hot dataset.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Shapley value, the attribution principle Shaver assigns to every embedding parameter."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the local-value Monte Carlo formulation that turns global Shapley values into per-instance permutation averages."},{"cited_title":"Shapley Value as Principled Metric for Structured Network Pruning","cited_arxiv_id":"2006.01795","evidence_quote":"Argues Shapley values satisfy null-player, symmetry, and efficiency properties desirable for pruning, motivating their use."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces single-shot pruning by connection sensitivity, the prior setting Shaver contrasts with and extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Presents single-shot embedding dimension search, a baseline and comparison point for Shaver's single-shot claim."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DeepFM backbone and second-order feature interactions that motivate non-zero placeholder values."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DCN-v2 and DCN-Mix backbones used in the main experiments."}],"review_version":1}