{"id":"f1c02b05-0e84-4ee0-87d0-916a0b216869","arxiv_id":"2608.12953","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SNIPER uses a 0/1 knapsack dynamic program for coarse depth pruning plus a fine-grained MLP width stage, achieving near-exact compression budgets and better average retention than six LLM pruners.","lead":"This paper introduces SNIPER, a two-stage method that prunes large language models by solving a knapsack problem to drop whole layers or blocks, then trims MLP columns to hit an exact size target. It reports that SNIPER reaches target compression ratios far more precisely than six prior pruners while keeping more task accuracy and lower variance across 18 benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SNIPER results imply Stage 1 violates the capacity constraint of Eq. 1: with α=32, observed CR 24.62% at target 25% means Stage 1 retained >C parameters, so the claimed conditional optimality is for an infeasible solution.","rationale":"The central claim has two parts: (i) the knapsack stage is conditionally optimal w.r.t. its own importance estimates, and (ii) budget adherence is near-exact. The reader's verdict correctly flags (i) as fragile because the importance estimates in Section 3.3 are computed on a model whose pruning configuration is set by the same DP (Section 5.4), a circular dependency the paper neither specifies nor proves convergent. I agree that this makes the 'fixed importance estimates' premise under-defined. However, there is a more direct, checkable failure: the DP is run on weights discretized by α (Section 3.2), so the recovered set can violate the true capacity C of Eq. 1. Table 8's own numbers confirm this—at target CR 25%, SNIPER's observed CR is 24.62% (retained 75.38% > 75%). Since Stage 2 only removes parameters, Stage 1 must have retained even more than 75.38%, so sum w(x) > C for the Stage 1 solution. The 'conditionally optimal allocation' is thus infeasible for the problem stated in Eq. 1. This holds regardless of how importance scores are obtained; it is a property of the rounded knapsack. The paper acknowledges the discretization trade-off in Sections 3.2 and 6.5, but the abstract and Section 3.2's optimality claim are not qualified accordingly. The fix is straightforward: either state the guarantee for the discretized problem (and verify feasibility w.r.t. the true budget), or use exact weights where tractable. The empirical evaluation is broad and the method is well engineered; with the above clarification the paper could still support its main empirical claims, which is why I keep the verdict conditional rather than moving to reject.","tokens_in":39658,"tokens_out":16464,"duration_ms":160258,"concrete_test":"Compute the true parameter count sum_{x in S} w(x) for the Stage 1 set S recovered by the α=32 DP on Qwen3-8B at target CR 25%, using exact non-rounded weights. If this sum exceeds C = 0.75 × total params (which is implied by the reported final CR of 24.62% and the fact that Stage 2 only removes parameters), then the solution is infeasible for Eq. 1 and the conditional-optimality guarantee fails. For a direct comparison, also solve the knapsack with α=1 (exact weights) on a small proxy model where exhaustive DP is tractable and check whether the α=32 selection is optimal and feasible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation 1 defines S* as the subset maximizing importance subject to sum_{x in S} w(x) ≤ C, where C is the target parameter budget. The DP in Section 3.2 solves this on weights discretized by α, i.e., with floor(w(x)/α) and capacity floor(C/α). Because weights are rounded down, the recovered set can have true weight exceeding C. Table 8 shows exactly this: at target CR 25%, SNIPER's observed CR is 24.62%, i.e., final retained parameters are 75.38% of the original, above the 75% budget. Since Stage 2 (fine-grained pruning) only removes parameters, Stage 1 must have retained even more than 75.38% of parameters, so the Stage 1 set violates Eq. 1's capacity constraint. Thus the 'conditionally optimal' allocation claimed in the abstract and Section 3.2 is not even feasible for the stated optimization problem. The paper acknowledges in Section 3.2 that increasing α trades off precision, but the central guarantee is stated without this caveat; with the chosen α=32 the solution is infeasible, so the optimality claim does not hold for the actual parameter budget. This is separable from the heuristic nature of importance estimates (also a concern, Sections 3.3 and 5.4): even granting fixed, correct v(xi), the DP output is not a feasible optimum of Eq. 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SNIPER, a two-stage structured pruning framework for LLMs. Stage 1 solves a 0/1 knapsack problem over coarse components (attention blocks, MLP blocks, and whole layers) using dynamic programming, with component importance estimated from logit divergence under iterative partial pruning. Stage 2 performs fine-grained MLP column pruning to fill the remaining parameter budget. The paper introduces the Compression Ratio Adherence Factor (CRAFT) as a metric for budget fidelity and evaluates SNIPER on four architectures, six baselines, and 18 tasks, reporting a mean rank of 1.25 and near-exact compression-ratio adherence.","tokens_in":39987,"tokens_out":9415,"duration_ms":95707,"significance":"If the central claims hold, SNIPER would be a practically useful alternative to greedy structured pruners: it offers a non-greedy selection mechanism, a simple budget-fidelity metric, and an unusually broad evaluation suite spanning diverse architectures and tasks. The DP recurrence is standard and the empirical comparison is extensive, including calibration sensitivity, importance-score transfer, and ablation studies. The conditional-optimality framing is honest in that it does not claim a global guarantee over all pruning configurations, and the paper explicitly concedes in Section 8 that no guarantee extends to importance estimation. However, the two concerns below affect load-bearing parts of the paper: feasibility of the claimed capacity constraint, and the circularity/under-specification of the iterative importance estimation loop.","major_comments":[{"comment":"The DP solution is not guaranteed to satisfy the capacity constraint of Eq. (1). The recurrence discretizes weights as floor(w(x)/alpha) and capacity as floor(C/alpha); rounding weights downward allows the reconstructed set to have true weight exceeding C. Table 8 shows this is not hypothetical: taking the compression ratio as the fraction of parameters removed, at target CR 25% SNIPER's observed CR is 24.62%, so the final model retains 75.38% of parameters while the budget allows 75%; since Stage 2 only removes parameters, Stage 1 must have already exceeded the budget. The same pattern occurs for every target CR in Table 8 (observed CR is below target in all five rows). Consequently, the statement that SNIPER yields a 'conditionally optimal' allocation with respect to Eq. (1) is not supported for the actual parameter budget; at best the solver is optimal for a discretized surrogate problem. The paper should either use a feasibility-preserving discretization (e.g., rounding weights up and capacity down), state the guarantee for the discretized problem and provide a feasible repair step, or explicitly re-define Eq. (1) in terms of the discretized weights.","section":"Section 3.2, Eq. (1); Table 8"},{"comment":"The importance scores v(x_i) in Eq. (3) are defined relative to a partially pruned model M^{(i-1)}, but Section 3.3 never specifies how M^{(i-1)} is constructed. Section 5.4 states that its pruning configuration is 'determined by our dynamic programming solver,' i.e., by the same S* that Eq. (1) computes from v. This makes the definition of v circular unless an explicit iterative fixed-point procedure (initialization, update rule, convergence criterion) is provided. Because the paper's optimality guarantee is explicitly conditional on fixed importance estimates, the missing specification means the guarantee does not currently apply to the actual algorithm used in the experiments. The ablation against leave-one-out scoring in Table 4 shows that the choice of scoring procedure materially changes results, so this is a central issue rather than a cosmetic one.","section":"Section 3.3 and Section 5.4"}],"minor_comments":[{"comment":"Cross-references are inconsistent: Section 3.2 cites 'Appendix 6.5' and Section 6.4 cites 'Appendix 6.2'; both should point to the corresponding sections (6.5 and 6.2, respectively) rather than to an appendix.","section":"Section 3.2; Section 6.4"},{"comment":"The headline CRAFT value of 0.98 appears in the abstract, Section 5.3, and Figure 1, but Table 8's SNIPER rows average to approximately 0.97. The paper should reconcile the reported number or indicate explicitly that 0.98 refers to the values in Table 7 rather than Table 8.","section":"Abstract; Section 5.3; Table 8"},{"comment":"The text states that at 35% compression on Qwen3-8B with RFT, SNIPER reduces task-wise variance to 15.90% 'compared to 17–30% for competing pruners,' but Table 1 reports 2SSP with Std RP 15.80% in the same configuration, which is lower than SNIPER's 15.90%.","section":"Section 5.1; Table 1"},{"comment":"The sentence 'Across all configurations except one, SNIPER achieves the highest average retention performance' is contradicted by the same paragraph: on LLaMA-3.1-8B-Instruct at 25% compression with RFT, both ReplaceMe (88.90) and ShortGPT (88.47) outperform SNIPER (87.34). The count should be corrected.","section":"Section 5; Table 1"},{"comment":"LAMBADA appears in both 'Generative Performance' and 'NLU and Inference' task lists. Please clarify whether it is evaluated once and categorized twice, or whether two different task formats (log-perplexity vs. next-word prediction accuracy) are used.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"Both major concerns are correctable in a revision: the DP feasibility guarantee needs to be made precise (or the claim restricted to a feasible surrogate), and the iterative importance-estimation loop needs an explicit specification. The empirical comparison is extensive and the CRAFT metric is a useful addition; I would not reject on the current evidence. I would also ask the authors to report repeated runs or seed information, since the robustness claims currently rest on single runs of a stochastic pruning/fine-tuning pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on SNIPER. The real contribution is a two-stage pruning scheme: a 0/1 knapsack DP over mixed-granularity transformer components (attention blocks, MLPs, full layers) followed by an MLP width-pruning stage that mops up residual budget. That combination is new for LLM pruning, and the evaluation is impressively broad—four architectures, 18 tasks, six baselines, with ablations on calibration, transferability, and the discretization factor. The CRAFT metric for budget adherence is simple and useful. If the results hold up, this is a meaningful practical win over greedy pruners, especially for exact-budget deployment.\n\nThe paper's empirical story is largely convincing. SNIPER ranks first in most configurations and has lower task variance than baselines, even when it loses on average retention (e.g., Llama-3.1 at 25% with RFT, where ReplaceMe and ShortGPT do better on average but have higher variance). The ablations show both stages matter and that the knapsack approach beats leave-one-out scoring. The authors also test transfer of importance scores across budgets, which is a nice practical touch.\n\nThe soft spots are real, and they center on the word 'optimal.' The stress-test note is correct: because the DP floors weights to multiples of alpha, the Stage 1 solution can exceed the true capacity C. Table 8 shows at target CR 25%, SNIPER's final observed CR is 24.62%, i.e., it retained 75.38% of parameters. Since Stage 2 only removes parameters, Stage 1 kept at least that many, so it violated the budget constraint of Eq. 1. The 'conditionally optimal' claim is therefore not about the actual budget. This is partly mitigated because Stage 2 fixes the final size and the violation is only ~0.4 percentage points, but the paper should state this explicitly when claiming conditional optimality.\n\nThe bigger concern is the importance estimation loop. Section 3.3 says importance is computed from a 'partially pruned model M(i-1)', and Section 5.4 says that configuration is 'determined by our dynamic programming solver.' That sounds circular: the scores depend on the selected set, which depends on the scores. The paper never specifies the iterative procedure precisely. Without that, the optimality guarantee is conditional on a poorly defined heuristic. The authors do concede in Section 8 that no guarantee extends to importance estimation, but the abstract and Section 3.2 should carry that caveat more prominently.\n\nMinor issues: no code release, single-seed runs for RFT, and some baselines are 're-engineered,' which could favor the proposed method. These are fixable in revision.\n\nBottom line: this deserves a serious referee. The empirical contribution is substantial and the knapsack angle is new. A referee should ask for (1) a precise statement of when the DP solution violates capacity and how Stage 2 compensates, (2) a detailed algorithm for the iterative importance estimation, and (3) code. I'd expect major revision, not rejection.","headline":"Useful two-stage knapsack-based pruner with broad empirical evaluation; the 'conditional optimality' claim needs tightening because the discretized DP can violate the stated budget and the importance loop is under-specified.","tokens_in":40512,"tokens_out":4863,"would_cite":true,"duration_ms":47111,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Structured pruning of large language models can be cast as a 0/1 knapsack problem, and the paper claims dynamic programming with a fine-grained width stage achieves near-exact budget adherence and better task-level stability than greedy…","keywords":["LLM compression","structured pruning","knapsack optimization","depth pruning","width pruning","budget adherence","importance estimation","CRAFT"],"falsifier":"Compare SNIPER's selected component set against a greedy depth pruner that uses the same iterative marginal importance scores on the same model and budget; if the greedy set matches or beats SNIPER's retention, the reported advantage is carried by the scores rather than by the knapsack solver, and the conditional-optimality claim would not be what drives the result.","tokens_in":39486,"feed_emoji":"🎯","tokens_out":7803,"duration_ms":71424,"temperature":0.7,"pith_summary":"This paper claims that structured pruning of LLMs should be treated as a global budget-allocation problem instead of a sequence of greedy local choices. It decomposes each transformer layer into coarse items — the attention block, the MLP block, or the whole layer — assigns each a parameter weight and a marginal importance value, and solves the resulting 0/1 knapsack with dynamic programming. A second, fine-grained stage prunes MLP columns to spend the leftover budget, which the paper credits for hitting target compression ratios almost exactly: a CRAFT of 0.98 where, by its measurements, baseline pruners deviate by up to 33%. Across four architectures and 18 tasks, SNIPER reports the best average retention in nearly every configuration, a mean rank of 1.25, and lower task-wise variance than six baselines. A sympathetic reader takes away that non-greedy optimization can turn pruning from a myopic heuristic into a budget-exact, conditionally optimal procedure.","feed_headline":"Knapsack solver prunes LLMs to budget with 0.98 fidelity","feed_subtitle":"Two-stage pruner replaces greedy heuristics and hits near-exact compression targets across four model families.","key_machinery":"The load-bearing object is a 0/1 knapsack instance built from a mixed-granularity decomposition of a transformer: each layer contributes three mutually exclusive items — attention block, MLP block, or whole layer — grouped so the solver picks at most one branch per layer. The dynamic programming recurrence $f(i,j)$ over the first $i$ items and capacity $j$ finds the maximum-value subset, and a discretization factor $\\alpha$ reduces the capacity dimension to keep runtime tractable. A second mechanism, the sensitivity score $\\Omega_j = |D_{j,:} \\cdot (G_{:,j} \\odot U_{:,j})|$ for each MLP column, tells the fine-grained stage which columns to prune after the coarse stage has left a residual capacity. Together these two mechanisms turn pruning into a global, budget-constrained selection problem followed by a precision fill step.","core_discovery":"The paper's central claim is that the depth-pruning step of an LLM pruner can be set up as Equation (1): choose the subset $S^* \\subseteq X$ of coarse components that maximizes total value $\\sum_{x \\in S} v(x)$ while keeping total parameter weight $\\sum_{x \\in S} w(x)$ at or below budget $C$. The claimed advance is not global optimality — the authors state plainly that the solution is optimal only with respect to fixed importance estimates — but it is a guarantee that greedy methods lack. The importance estimates themselves are computed iteratively as the marginal logit divergence of removing each component from a partially pruned model, so the scores are meant to reflect joint effects rather than isolated component quality. The follow-up width stage distributes the remaining budget across MLP layers in inverse proportion to their importance and removes the least sensitive columns, which is how the method achieves near-exact budget adherence while preserving tensor structure. The paper also introduces CRAFT, the ratio of observed to target compression, as a way to quantify this budget fidelity.","pith_inferences":["Beyond the paper: the observed transferability of importance scores across budgets suggests the same scores could be reused across checkpoints of a model family, making the dominant scoring cost a one-time investment per family.","Beyond the paper: because attention and MLP blocks enter the knapsack as exchangeable items, the same solver could be pointed at expert-level components in mixture-of-experts models or at non-uniform per-layer width budgets without changing the core formulation.","Beyond the paper: the strong overlap of pruned layers across budgets, which the paper presents as interpretable salience, could be read as a diagnostic tool for architectural redundancy that is independent of any particular compression target."],"forward_implications":["If the central claim holds, structured pruning becomes budget-exact: a practitioner who asks for a 25% or 35% model gets a model that actually has that parameter count, removing the capacity slack the paper documents in existing pruners.","The conditional-optimality guarantee means that, for the importance scores SNIPER computes, no other subset of the same coarse components within the budget has higher total value; greedy depth pruners offer no comparable guarantee.","Because the importance scores transfer across compression ratios with only a small retention loss, the expensive scoring pass can be amortized over multiple target budgets rather than recomputed from scratch.","The framework's performance on dense, reasoning-tuned, fused-MLP, and mixture-of-experts architectures suggests the knapsack formulation is not bound to one transformer design.","Lower task-wise standard deviation across 18 tasks implies the method is less likely to over-optimize one metric at the expense of others, addressing a documented failure mode of greedy pruning."],"supporting_citations":[{"why":"Supplies the 0/1 knapsack formulation and dynamic-programming solution that Stage 1 of SNIPER is built on.","marker":"[48]"},{"why":"2SSP is the dual-axis baseline whose two-stage design SNIPER compares against and whose budget distribution motivates the fine-grained stage.","marker":"[51]"},{"why":"ReplaceMe is a depth-pruning baseline that replaces contiguous blocks with a learned linear projection; SNIPER compares retention against it.","marker":"[12]"},{"why":"ShortGPT is the layer-redundancy depth-pruning baseline that provides the coarse component removal comparison point.","marker":"[13]"},{"why":"SLEB is the leave-one-out block-elimination baseline whose scoring strategy SNIPER ablates as SNIPER-LeaveOneOut.","marker":"[14]"},{"why":"LLM-Pruner is the dependency-graph width-pruning baseline that SNIPER must outperform on retention and variance.","marker":"[16]"},{"why":"SliceGPT is the PCA-based row and column width-pruning baseline whose irregular tensor shapes motivate SNIPER's mixed-granularity stage.","marker":"[18]"},{"why":"Wanda's activation-aware scoring is the alternative column-selection heuristic used in the SNIPER-ACT ablation.","marker":"[33]"},{"why":"Magnitude-based pruning is the alternative column-selection heuristic used in the SNIPER-MAG ablation.","marker":"[32]"}],"fun_headline_variants":["Knapsack-based LLM pruning hits 0.98 budget fidelity","SNIPER: knapsack optimization prunes LLMs to exact budget","Binary knapsack trims LLMs with near-perfect compression","LLM pruning via knapsack: beats SOTA with 0.98 fidelity","Knapsack solver for LLM pruning: exact budget, top rank"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The knapsack's optimality is conditional on the importance scores being correct, and those scores are heuristic and path-dependent; if the scores fail to capture true performance loss, the optimal selection is optimal for the wrong objective.","fun_headline_variants_meta":{"raw":{"variants":["Knapsack-based LLM pruning hits 0.98 budget fidelity","SNIPER: knapsack optimization prunes LLMs to exact budget","Binary knapsack trims LLMs with near-perfect compression","LLM pruning via knapsack: beats SOTA with 0.98 fidelity","Knapsack solver for LLM pruning: exact budget, top rank"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00038,"raw_usage":{"total_tokens":2027,"prompt_tokens":960,"completion_tokens":1067,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":968}},"tokens_in":576,"tokens_out":1067,"duration_ms":9037,"temperature":1.0,"reasoning_tokens":968,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:50:31.365825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare SNIPER's selected component set against a greedy depth pruner that uses the same iterative marginal importance scores on the same model and budget; if the greedy set matches or beats SNIPER's retention, the reported advantage is carried by the scores rather than by the knapsack solver, and the conditional-optimality claim would not be what drives the result.","supporting_citations":[{"cited_title":"Knapsack problems,","cited_arxiv_id":null,"evidence_quote":"Supplies the 0/1 knapsack formulation and dynamic-programming solution that Stage 1 of SNIPER is built on."},{"cited_title":"2SSP: A two-stage framework for structured pruning of LLMs,","cited_arxiv_id":null,"evidence_quote":"2SSP is the dual-axis baseline whose two-stage design SNIPER compares against and whose budget distribution motivates the fine-grained stage."},{"cited_title":"Replaceme: Network simplification via depth pruning and transformer block linearization,","cited_arxiv_id":null,"evidence_quote":"ReplaceMe is a depth-pruning baseline that replaces contiguous blocks with a learned linear projection; SNIPER compares retention against it."},{"cited_title":"SLEB: Streamlining LLMs through redundancy verification and elimination of transformer blocks,","cited_arxiv_id":null,"evidence_quote":"SLEB is the leave-one-out block-elimination baseline whose scoring strategy SNIPER ablates as SNIPER-LeaveOneOut."},{"cited_title":"Llm-pruner: on the structural pruning of large language models,","cited_arxiv_id":null,"evidence_quote":"LLM-Pruner is the dependency-graph width-pruning baseline that SNIPER must outperform on retention and variance."}],"review_version":1}