{"id":"2b588ebe-2b2d-4e40-80aa-959088257803","arxiv_id":"2501.08631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SWSC combines channel K-means clustering with an SVD low-rank error correction to compress LLM weights, and reports lower perplexity than RTN quantization on Llama-2-7B Q and K projections at 2 to 3 average bits.","lead":"SWSC compresses LLM weight matrices by grouping similar channels with K-means and replacing each cluster with a shared mean vector, then adding a low-rank SVD error correction. The paper reports that this beats simple rounding (RTN) on Llama-2-7B attention query and key matrices at 2 to 3 average bits, but it does not compress the full model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central low-precision claim is only tested on Q/K projections; Section IV-B excludes V, leaving >80% of Llama-2-7B at full precision, so '2-bit' is per-matrix, not per-model.","rationale":"I agree with the reader that the mechanism is plausible and the reported Q/K results, if bit-matched, are non-trivial. The soft spot I find is broader than the reader's 'Q/K insensitivity' framing: even granting that Q/K are insensitive, compressing only them does not produce a low-precision LLM. The paper itself concedes V is excluded. For Llama-2-7B, attention Q and K are a minority of parameters; MLPs and embeddings dominate. A method that cannot touch those matrices at 2 bits cannot deliver the abstract's central promise. My proposed concrete test would settle this by measuring full-model perplexity and the true whole-model bit rate. I therefore keep the reader's CONDITIONAL verdict: the paper needs either a scoped claim or full-model evidence. If the proposed test fails, the verdict should move to REJECT.","tokens_in":7156,"tokens_out":10350,"duration_ms":106613,"concrete_test":"Run SWSC at 2 average bits on (a) V alone, (b) MLP alone, and (c) all linear/embedding matrices (Q, K, V, O, gate/up/down, embeddings) in Llama-2-7B, then measure WikiText-2 perplexity and the true whole-model average bit rate. If V-only or MLP-only compression degrades perplexity well beyond the Q/K-only case, or if the full-model 2-bit setting diverges, the reported results cannot support a low-precision compressed LLM claim. Also report the exact k and r values and include cluster-label storage in the bit count.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-B restricts SWSC to Q and K projectors and explicitly leaves V uncompressed: 'the Value Projector stores the specific features of the model and has a higher requirement for accuracy, so it is not compressed.' The Abstract and Conclusion, however, claim that SWSC can 'ensure that the performance of the compressed LLM is not significantly affected even under low-precision conditions.' The experimental support (Table I) is a per-matrix 'Avg. Bits' figure, not a whole-model bit rate. In Llama-2-7B, Q and K are roughly 1.07B of ~6.7B parameters (~16%), so even at 2 bits for those matrices the whole model runs at ~13.8 average bits. The load-bearing premise that V and the MLP/embedding matrices can be passed over is not an established property; it is asserted without an ablation, and if it is false the central claim fails for a genuinely compressed model. This is an over-claiming of the experimental scope, not an internal contradiction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SWSC, a post-training compression scheme for LLM weight matrices. The method clusters the channels of a weight matrix with k-means, replaces each cluster by its mean, and stores a cluster-label vector plus the cluster representative vectors; to recover accuracy, it computes the residual matrix W - W' and stores a truncated SVD of that residual, which is added back at inference time. On Llama-2-7B, the authors compress the query and key projectors (separately and jointly) to what they call 2-3 average bits and report WikiText-2 perplexities of roughly 6.5-10.9, compared with RTN at the same bit budget, which gives values such as 4958.4 or NaN. The paper's central claim is that SWSC can keep the compressed LLM's performance largely intact even under low-precision conditions, and that it is a promising alternative to quantization.","tokens_in":7297,"tokens_out":5429,"duration_ms":56922,"significance":"If the per-matrix results are taken at face value, the paper demonstrates a simple and interesting compression idea: the method has no fitted constants or data-dependent normalizations, and the SVD error correction is computed directly from the actual compression error, so the comparison with RTN is not circular. This is a genuine strength. The reported numbers for Q/K compression at 2 bits are strikingly better than RTN and suggest that cluster-then-correct is worth exploring. However, the significance as a full-model LLM compression method is not yet established: only the Q and K projectors are compressed, the average-bit accounting does not state the storage precision of the stored values, and no state-of-the-art PTQ baseline is included. The paper's general claims in the Abstract and Section V go beyond the experimental scope.","major_comments":[{"comment":"The experimental protocol compresses only the Query and Key projectors, and Section IV-B explicitly states that the Value Projector is not compressed. In Llama-2-7B, Q and K together are roughly 1.07B of about 6.7B parameters, so the 'Avg. Bits' figures in Table I are per-matrix, not per-model. If Q/K are at 2 bits and all remaining weights stay at 16 bits, the whole model runs at roughly 13.8 average bits, not 2 bits. The Abstract and Section V claims that SWSC 'can effectively ensure that the performance of the compressed LLM is not significantly affected even under low-precision conditions' therefore outrun the evidence. Please either report whole-model bit rates and include an ablation that compresses V (and ideally the MLP and embedding matrices), or scale the claims to 'Q and K projectors at low precision.'","section":"§IV-B, Table I, Abstract/V"},{"comment":"The definition of 'Avg. Bits' is incomplete. Section IV-C gives only the cluster/rank increments in Table II, but it does not state the storage precision of the cluster representative vectors, the cluster-label vector, or the SVD factors U_rΣ^{1/2} and Σ^{1/2}V_r. The reported 0.5-bit increments correspond to storing 16-bit values: for m=4096, one additional 4096-dimensional representative per 128 clusters contributes 128×4096×16/4096² = 0.5 bits per original weight. If fp32 or int8 storage were used, all average-bit figures would change; label overhead also needs to be included. Because RTN is an integer-quantization method, the comparison requires a precise bit-accounting convention. Please state the storage format for every stored quantity and recompute Table I under that convention.","section":"§III-C and §IV-C"},{"comment":"Table I contains no row for the unmodified Llama-2-7B perplexity on WikiText-2. The central claim that SWSC 'ensures that the performance of the compressed LLM is not significantly affected' cannot be evaluated without this baseline. For example, the SWSC 2-bit Q&K result of 10.886 may be far from the full-precision model's perplexity even though it is much better than RTN's 10490. Please add the uncompressed baseline perplexity and, if possible, a 4-bit RTN row so that the reader can see where the compressed model stands relative to the original.","section":"Table I"},{"comment":"The comparison is limited to RTN, and the paper does not compare with standard post-training quantization methods such as GPTQ, AWQ, or SmoothQuant, even though these are cited in the related-work section. This matters because the conclusion states that SWSC is 'a promising approach for LLM compression' in general, not merely better than RTN. Please add at least one state-of-the-art PTQ baseline at comparable bit rates, or temper the conclusion to a claim about outperforming RTN. In addition, the statement that the Value Projector 'stores the specific features' and therefore should not be compressed is asserted without quantitative support; an ablation showing the relative sensitivity of Q, K, and V under SWSC would make the selection criterion credible.","section":"§IV-A and §IV-B"}],"minor_comments":[{"comment":"The statement that 'a compression rate of over 90% can be achieved' for m=4096 and k=256 appears to count only the cluster-representative storage and not the SVD error-compensation factors; with nonzero retained rank the compression rate is lower. Please clarify the accounting in this sentence.","section":"§III-B"},{"comment":"The 'nan' entry for RTN on the K projector at 2 bits is unexplained; state whether it is a numerical overflow, a divergence, or a failed evaluation.","section":"Table I"},{"comment":"The column header 'Avg. Bits' should specify 'average bits per original weight parameter,' and the sentence about increments of 0.5 bits should explicitly state the assumed bit width of the stored real-valued factors.","section":"§IV-C"},{"comment":"There are several typographical and formatting issues: the 'Index Terms' line is broken, some inline math symbols are mangled in the extracted text, and the RTN rows in Table I are visually ambiguous. A careful proofread would improve readability.","section":"Various"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and the narrow per-matrix comparison with RTN is credible, but the manuscript currently overclaims the experimental scope. The missing bit-precision accounting and the absence of a full-model compression test are the two issues that most need attention. I would not reject on novelty grounds: the combination of k-means channel sharing with an SVD residual correction is a reasonable contribution to the compression toolbox, and the result could be publishable after a substantial revision that either broadens the experiments or narrows the claims. The paper's current form is closer to a workshop-level report than a full archival paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new bit here is the specific pipeline: channel-wise K-means clustering of weight matrices plus a truncated-SVD correction on the residual. That's a legitimate combination of product quantization and low-rank error compensation, and the paper deserves credit for stating the procedure clearly and for reporting a fair narrow comparison: at matched average bits, SWSC beats RTN on Llama-2-7B's Q and K projections on WikiText-2. No circularity, no fitted constants; the SVD error matrix is computed from actual compression error.\n\nThe soft spots are in the gap between the evidence and the claim. The abstract and conclusion say SWSC keeps 'the compressed LLM' unharmed at low precision, but Section IV-B compresses only Q and K projectors and explicitly leaves V uncompressed, not to mention MLPs and embeddings. On Llama-2-7B that's roughly 16% of the parameters, so the whole model runs at something like 13.8 average bits, not 2. The sensitivity premise about Q/K vs V is asserted, not shown. Also: one model, one dataset, one baseline (RTN), no error bars, no code, and the bit accounting in Table II doesn't state how many bits are used for the cluster representatives, labels, and SVD factors; without that the 'Avg. Bits' number is not fully interpretable. I'd want to see an ablation that compresses V or the full model, and a comparison to at least one stronger PTQ baseline (e.g., GPTQ or a plain product-quantization baseline) before trusting the central claim.\n\nWho is this for? People working on post-training compression will find the recipe usable and the idea of SVD error compensation for clustering worth borrowing. As a paper, it's honest incremental work with an overreaching title, not a flawed mechanism.\n\nRecommendation: send it to peer review but with a request for a major revision that either narrows the claims to per-matrix compression or extends the experiments to full-model compression. A serious referee would be able to work with this.","headline":"Reasonable compression recipe for Q/K projections with a clear narrow win over RTN, but the 'low-precision LLM' claim overreaches the evidence.","tokens_in":7870,"tokens_out":1995,"would_cite":false,"duration_ms":19539,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims SWSC, a channel-clustering plus truncated-SVD compression method, holds Llama-2-7B's WikiText-2 perplexity near 7-10 at 2 average bits for query and key projectors, while RTN quantization fails at the same budget.","keywords":["large language model compression","channel clustering","K-Means","singular value decomposition","low-precision weights","perplexity","attention projector","weight sharing"],"falsifier":"Run SWSC on the value projector of Llama-2-7B at 2 average bits and measure WikiText-2 perplexity; if it degrades as sharply as RTN (or worse), the method's success is limited to the projectors it already excludes. Alternatively, apply SWSC to every linear layer in the model and see whether the overall perplexity stays near the uncompressed value.","tokens_in":6898,"feed_emoji":"🧠","tokens_out":5663,"duration_ms":51946,"temperature":0.7,"pith_summary":"This paper proposes SWSC, a post-training compression method that treats a weight matrix as a set of channels, clusters similar channels with K-Means, and stores one shared representative vector per cluster instead of every channel. To keep the approximation from hurting accuracy, it computes the error matrix between the original and reconstructed weights and adds back a truncated singular-value-decomposition version of that error. On the query and key projectors of Llama-2-7B, the method holds WikiText-2 perplexity between roughly 6.5 and 10.9 at average bit budgets of 3 and 2, where the rounding-to-nearest (RTN) baseline produces perplexities in the thousands or a NaN. The paper concludes that SWSC can compress attention-projector weights to very low precision without significantly damaging performance, and positions the method as orthogonal to existing quantization, pruning, distillation, and low-rank approaches.","feed_headline":"Clustering similar weight channels keeps LLM quality at 2 bits","feed_subtitle":"On Llama-2-7B, SWSC preserves near-7 perplexity where rounding-to-nearest collapses to NaN or thousands.","key_machinery":"The central mechanism is channel-wise K-Means clustering of the weight matrix together with truncated SVD error compensation. A weight matrix $W \\in \\mathbb{R}^{m\\times m}$ is clustered along its $m$ channels; each channel is assigned a cluster label, and every channel in a cluster is replaced by that cluster's mean vector, reducing storage to roughly $(k+1)m$ parameters when there are $k$ clusters. The error matrix $W_{\\text{err}} = W - W'$ is then decomposed as $U\\Sigma V^T$, and only the top $r$ singular values and vectors are kept, stored as $U_r\\Sigma_r^{1/2}$ and $\\Sigma_r^{1/2}V_r$; during inference these are added to $W'$. The cluster labels, representative vectors, and truncated factors together determine the advertised bit rate, with the paper reporting that each increment of 128 clusters or 64 retained singular ranks adds 0.5 average bits for Llama-2-7B's self-attention layer.","core_discovery":"The central discovery is that a channel-wise clustering plus error-SVD scheme can keep a large language model's behavior nearly intact at bit budgets where standard low-precision rounding disintegrates, at least for the query and key projections of the self-attention layer. The evidence is Table I: with 2 average bits, SWSC gives perplexities of 7.297 for the query projector, 7.148 for the key projector, and 10.886 for both together on WikiText-2, while RTN at the same budget reaches 4958.396 for the query projector and returns 'nan' for the key projector. The paper presents this as evidence that query and key projector weights are insensitive to their exact values, so shared cluster representatives are acceptable, whereas the value projector is deliberately excluded because it 'stores the specific features of the model' and requires higher accuracy.","pith_inferences":["If the query/key insensitivity generalizes beyond Llama-2-7B, SWSC could be applied to attention projections in other transformer LLMs, but the paper does not test cross-model transfer.","The success at 2 bits suggests attention-projector weights have a clustered latent structure that standard rounding ignores, which could motivate further analysis of when mean-replacement is safe.","The error correction acts as a low-rank residual on a codebook approximation, so combining SWSC with low-rank adaptation or pruning may yield smaller footprints than either approach alone, though the paper does not investigate this.","A head-to-head against stronger post-training quantizers such as GPTQ would be needed to show SWSC's advantage is not limited to the weak RTN baseline."],"forward_implications":["SWSC can be combined with existing quantization, pruning, knowledge distillation, or low-rank methods, since it is presented as an orthogonal weight-compression technique.","At very low average bits, SWSC is more stable than RTN on attention-projector weights, so it offers an alternative deployment path for memory-constrained devices.","The cluster count $k$ and retained rank $r$ give a direct control knob for trading compression ratio against model quality.","The paper's stated next step is to combine SWSC with other orthogonal compression methods, which would test whether the gains stack on top of standard quantization.","Because the current experiments cover only query and key projectors, the method's broader applicability to other weight matrices remains open even if the reported numbers hold."],"supporting_citations":[{"why":"Product quantization, the technique that motivates channel-wise clustering and codebook storage in SWSC.","marker":"[21]"},{"why":"Singular value decomposition, the mathematical tool used to build the low-rank error-compensation term.","marker":"[5]"},{"why":"Llama-2-7B, the model on which the compression and perplexity experiments are run.","marker":"[22]"},{"why":"WikiText-2, the dataset used to measure perplexity of the compressed model.","marker":"[23]"}],"fun_headline_variants":["Channel clustering plus error-SVD keeps LLM afloat at 2-bit","Shared-weight clustering preserves LLM at 2-bit, RTN fails","SWSC: Cluster similar channels, SVD repair, keep perplexity low","2-bit LLM that survives: SWSC beats rounding-to-nearest","Channel-wise clustering and SVD keep LLM at 2-bit precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that query and key projector weights are insensitive to their exact values, so replacing whole channels with cluster means plus a low-rank correction preserves model behavior, while value projectors are too sensitive and are excluded.","fun_headline_variants_meta":{"raw":{"variants":["Channel clustering plus error-SVD keeps LLM afloat at 2-bit","Shared-weight clustering preserves LLM at 2-bit, RTN fails","SWSC: Cluster similar channels, SVD repair, keep perplexity low","2-bit LLM that survives: SWSC beats rounding-to-nearest","Channel-wise clustering and SVD keep LLM at 2-bit precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000597,"raw_usage":{"total_tokens":2763,"prompt_tokens":885,"completion_tokens":1878,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":1780}},"tokens_in":501,"tokens_out":1878,"duration_ms":15039,"temperature":1.0,"reasoning_tokens":1780,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:21:27.664347+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SWSC on the value projector of Llama-2-7B at 2 average bits and measure WikiText-2 perplexity; if it degrades as sharply as RTN (or worse), the method's success is limited to the projectors it already excludes. Alternatively, apply SWSC to every linear layer in the model and see whether the overall perplexity stays near the uncompressed value.","supporting_citations":[{"cited_title":"Numerical inverting of matrices of high order,","cited_arxiv_id":null,"evidence_quote":"Singular value decomposition, the mathematical tool used to build the low-rank error-compensation term."}],"review_version":1}