{"id":"8a827344-7367-4352-ac0f-dc22db5ab4fc","arxiv_id":"2506.05432","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PCDVQ compresses LLM weights to 2 bits by quantizing vector directions and magnitudes separately with distribution-matched codebooks, reporting modest zero-shot accuracy gains over prior vector quantization baselines.","lead":"This paper proposes PCDVQ, a method for compressing large language models to about 2 bits per weight by quantizing the direction and the magnitude of weight vectors separately instead of together. The authors report roughly 1.5 points higher zero-shot accuracy than existing 2-bit methods, which matters for running big models on phones and edge devices.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 'at least 1.5%' gain is unsupported because the '2.125-bit' PCDVQ row actually costs 2.25 bits/weight, so it is compared against baselines at lower bit rates.","rationale":"The reader's weakest_assumption focuses on whether codebooks remain calibrated after randomized Hadamard regularization. That is a legitimate transfer concern, but the paper's own guarantee for this step is standard in the QuIP/QuIP# literature, and the spherical symmetry of a random orthogonal transform is a strong theoretical basis: for large column dimensions, subvectors of a rotated vector are nearly uniform in direction and their magnitudes are nearly root-chi-square. The empirical results on six models also suggest that any residual miscalibration is not catastrophic. By contrast, the bit-rate arithmetic error directly attacks the central quantitative claim. The abstract promises 'at least 1.5%' improvement at 2-bit level, and the experiments table the PCDVQ 2.125 result as the main evidence. If that result is actually at 2.25 bits/weight, the comparison is unfair and the margin over equal-bit baselines is smaller and sometimes below 1.5%. This is a concrete, checkable error in the paper's own Appendix A.3. I do not change the reader's conditional verdict because the decoupling idea remains plausible and the bit accounting can be corrected and re-tested, but the condition should explicitly require matched-bit comparisons and a re-computed headline margin.","tokens_in":14765,"tokens_out":12136,"duration_ms":145645,"concrete_test":"Re-run the LLaMA-2-7B, LLaMA-2-13B, LLaMA-3-8B, and Mistral-7B evaluations under three PCDVQ configurations: (a=14,b=2) at true 2.0 BPW, (a=15,b=2) at true 2.125 BPW, and (a=16,b=2) at true 2.25 BPW. Compare each configuration only against baselines with the same actual bits-per-weight, especially VPTQ 2.0/2.02, VPTQ 2.26, and GPTVQ 2.25. If the average zero-shot margin over these matched-bit baselines is below 1.5 percentage points or is not consistently positive, the headline claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is not codebook calibration but the bit-rate arithmetic behind the headline gain. Appendix A.3 states that PCDVQ at '2.125-bit' sets a=16, b=2, k=8, and computes BPW=(a+b)/k=2.125. But (16+2)/8=2.25. Thus the PCDVQ rows labeled 2.125 in Tables 1 and 2 actually cost 2.25 bits/weight, the same budget as GPTVQ 2.25 and VPTQ 2.26, not the 2.0-2.1-bit budgets of most baselines. At matched budgets the claimed 'at least 1.5%' margin shrinks: for LLaMA-2-7B, PCDVQ 60.44 vs VPTQ 2.26 59.36 is +1.08; for LLaMA-2-13B, PCDVQ 63.66 vs VPTQ 2.18 63.11 is +0.55; for LLaMA-3-70B, PCDVQ 71.98 vs VPTQ 2.07 70.74 is +1.24; for Mistral-7B, PCDVQ 64.33 vs VPTQ 2.04 63.20 is +1.13. No PCDVQ row at actual 2.0 bits is reported, so the '2-bit' version is untested. This is a concrete numerical error, not a stylistic issue: if the implemented configuration was really a=16, the experimental comparisons must be re-baselined at equal bit budgets; if the intended configuration was a=15, the experiments must be rerun because the direction codebook size changes by a factor of two and accuracy will change.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PCDVQ, a vector-quantization scheme for LLM weights that decouples each vector into direction and magnitude, quantizes these components independently with separate codebooks, and allocates more bits to direction. The motivation is an observed higher sensitivity of direction to quantization, and the method adds distribution-aligned codebooks: greedy sampling of E8 lattice directions for the direction codebook and Lloyd-Max scalar quantization for a root-chi-square magnitude distribution. Experiments on LLaMA-2, LLaMA-3, and Mistral-7B report lower perplexity and higher zero-shot accuracy than SQ/VQ baselines, with an advertised improvement of at least 1.5% zero-shot accuracy at the 2-bit level.","tokens_in":15127,"tokens_out":7177,"duration_ms":80040,"significance":"If the claims hold, PCDVQ would be a useful 2-bit weight-compression method and would support a clear design principle: allocate more bits to direction and avoid Euclidean-distance coupling in vector quantization. The analytical derivation of the magnitude distribution and the offline codebook construction are elegant, and the experimental evaluation spans six LLMs and multiple zero-shot tasks. However, the central quantitative claim is currently tied to incorrect bit arithmetic and unequal-budget comparisons, and the distribution-alignment premise behind DACC is cited rather than verified on the evaluated models. The core idea is promising, but the paper needs substantial revision before the headline claim can be accepted.","major_comments":[{"comment":"The BPW computation in Appendix A.3 is incorrect: with a=16, b=2, and k=8, (16+2)/8 = 2.25, not 2.125. The rows labeled 'PCDVQ 2.125' therefore consume the same budget as GPTVQ 2.25 and are compared against baselines at lower bit rates. Recomputing the advertised margin at matched budgets against VPTQ gives +1.08 for LLaMA-2-7B (60.44 vs 59.36), +0.55 for LLaMA-2-13B (63.66 vs 63.11), +1.24 for LLaMA-3-70B (71.98 vs 70.74), and +1.13 for Mistral-7B (64.33 vs 63.20), all below the claimed 'at least 1.5%'. The headline claim in the abstract and Section 1 is therefore unsupported by the reported data. Please either rerun the intended a=15 configuration, which changes the direction codebook size by a factor of two and will likely change accuracy, or re-baseline all comparisons at equal bit rates and adjust the claim accordingly. The memory-reduction percentages in Section 4.4 should also be recomputed for the true 2.25 BPW.","section":"A.3, Tables 1 and 2"},{"comment":"The main result tables contain only the mislabeled 2.125-bit PCDVQ rows; the 2.0-bit PCDVQ configuration (a=14, b=2) is reported only in Table 3 for LLaMA-2-7B. Consequently, the paper's repeated claim of outperforming baselines 'at 2-bit level' is not directly demonstrated for the other model and task combinations, and the comparisons in Tables 1 and 2 are at unequal bit rates. Please include 2.0-bit PCDVQ rows in the main tables, or explicitly restrict the claim to the evaluated bit rates.","section":"4.1 and Tables 1, 2"},{"comment":"The DACC codebooks are constructed from the assumption that after randomized Hadamard transform and per-column scaling, every weight column is approximately standard Gaussian, so directions are spatially uniform and squared magnitudes follow a chi-square distribution. This premise is cited to QuIP [5] but is not verified on any of the six models evaluated here. Since the E8 direction codebook and the Lloyd-Max magnitude codebook are precomputed once and reused for all models, residual correlations, outliers, or non-Gaussian column distributions would systematically miscalibrate them and could remove the claimed advantage. Please add a direct empirical check on the actual transformed weights, such as comparing the empirical distribution of r^2 to chi-square(k) or including an ablation with data-driven codebooks, to support the transferability claim.","section":"3.2.1 and 3.2.3"},{"comment":"There is an unresolved dimensionality mismatch in the direction codebook. The quantized direction parameter phi is defined as a (k-1)-dimensional tuple of angles in Eqs. (6)-(7), while the E8 lattice lives in R^8 and its directions are 8-dimensional unit vectors; for k=8 the codebook C_phi is said to have rows in R^(k-1)=R^7. The paper does not state how E8 directions are transformed into this angular space, nor how cosine similarity is computed between two angle tuples. Without this mapping the method is not reproducible and the claim that the E8 codebook is aligned with the direction distribution cannot be checked. Please specify the exact representation, such as converting 8D unit vectors to angular coordinates or quantizing the 8D normalized vectors directly and reconciling the codebook shape.","section":"3.2.3, Eq. (7)"}],"minor_comments":[{"comment":"Please fix typographical issues: 'access' should be 'assess' in Sections 1 and 3.1.2, and 'Llyod-max' should be 'Lloyd-Max' in Algorithms 2 and Table 4.","section":"Throughout"},{"comment":"The notation for the codebook size is inconsistent: C is written as R^{2n x k} but the number of codebook entries is later described as 2^n; use a single symbol for the bit-width, such as b, throughout.","section":"Section 2.2"},{"comment":"The definition of Delta r as sqrt((||v||-||v_hat||)^2) is redundant; defining Delta r = ||v|| - ||v_hat|| would make the decomposition of the squared Euclidean distance clearer.","section":"Section 3.1.2, Eq. (5)"},{"comment":"The direction-MSE definition 2||v||^2(1-cos theta) is introduced only in the ablation study; define it earlier in Section 3.1 so that the decomposition used in Figure 3 is available when the motivation is discussed.","section":"Section 4.3, Figure 3"},{"comment":"The efficiency analysis reports generation throughput on an RTX 4090 but does not measure dequantization latency or the overhead of the Hadamard transform mentioned in Appendix A.4; please clarify whether the reported 95.7 tokens/second includes these costs.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The bit-accounting error is the central issue: the advertised 1.5% margin disappears at matched bit rates, and the missing 2.0-bit rows leave the '2-bit level' claim unsupported. If the authors rerun the experiments or re-baseline honestly, the paper may become publishable. There is also no code or model-release link, which makes the E8-to-angular mapping ambiguity harder to resolve. The fit with cs.LG is appropriate, and I would not reject on novelty grounds; the direction/magnitude decoupling is a reasonable contribution, though the related PolarQuant KV-cache papers [13, 34] should be clearly differentiated in scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. The core idea—splitting weight vectors into direction and magnitude, spending most bits on direction, and building codebooks from the assumed Gaussian structure—is genuinely new and plausible. The headline result, however, is overstated because the '2.125-bit' PCDVQ configuration actually costs 2.25 bits/weight.\n\nWhat's new: the sensitivity analysis in Figure 1 is a useful observation, polar decoupling for weights (both PolarQuant papers are KV cache), and the distribution-aligned codebooks (greedy E8 directions, Lloyd-Max on the root chi-square) are a clean way to avoid fitting codebooks on evaluation data. The experiments span six models, and the ablations show the codebook choices matter.\n\nThe arithmetic error is concrete: Appendix A.3 says (16+2)/8 = 2.125, but it's 2.25. So every PCDVQ row in Tables 1 and 2 is actually at 2.25 bits, compared against baselines at 2.0–2.1 bits. At matched budgets the margin shrinks to around 0.5–1.2 points, not the claimed 1.5+. The 2-bit version (a=14,b=2) appears only in the ablation on one model, so the '2-bit' claim is not solidly established. Other issues: Eq. 9 misstates the angular marginals—the angles of a uniform sphere are not uniformly distributed except in the last coordinate; Algorithm 1's candidate set from E8 is underspecified; the abstract's sensitivity numbers (46.5% vs 2.3%) don't match the introduction's (30% vs 3%). These are all fixable.\n\nThe core method is not circular: codebooks are built from standard distributions, not fit to the eval tasks. The direction/magnitude sensitivity gap is a real phenomenon, and spending bits on direction is a sensible strategy. So this isn't a broken paper; it's a promising one that needs a corrected comparison.\n\nWho's this for? People working on extreme low-bit weight-only quantization, especially edge deployment. A serious referee should see it—the idea deserves scrutiny, but the paper needs major revisions before acceptance. I'd want matched-bit comparisons and a fixed bit budget.","headline":"The polar-decoupling idea is genuinely worth a look, but the headline 'at least 1.5%' gain rests on a bit-rate arithmetic slip and a missing matched-budget comparison.","tokens_in":15730,"tokens_out":3459,"would_cite":false,"duration_ms":37675,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that in 2-bit vector quantization of LLM weights, direction is far more sensitive than magnitude, so splitting the two and spending more bits on direction improves zero-shot accuracy by at least 1.5% over prior methods.","keywords":["vector quantization","LLM weight compression","post-training quantization","polar coordinates","codebook design","E8 lattice","Lloyd-Max quantization","2-bit quantization"],"falsifier":"Take a 2-bit PCDVQ run on LLaMA-2-7B, record the actual magnitudes of rotated weight vectors, and compare their quantiles with the root chi-square cumulative distribution used to build the Lloyd-Max codebook; a systematic mismatch would show that the standard-Gaussian premise fails. Equally decisive: replace the global E8 and Lloyd-Max codebooks with per-layer cluster centers at the same bit budget and check whether the reported 1.5% advantage survives.","tokens_in":1591,"feed_emoji":"🧭","tokens_out":1962,"duration_ms":71427,"temperature":0.7,"pith_summary":"The paper sets out to show that the dominant error in 2-bit vector quantization of LLM weights comes from direction, not length. It reports that separately clustering the directions of LLaMA-2-7B weights costs 46.5% zero-shot accuracy, while clustering magnitudes alone costs 2.3%, and explains that Euclidean distance weights magnitude error quadratically. To fix this imbalance, it converts weight vectors to polar coordinates, quantizes direction and magnitude with separate codebooks, and gives most of the bit budget to direction. The resulting method, PCDVQ, is claimed to beat previous 2-bit quantizers by at least 1.5% average zero-shot accuracy on six LLMs. If true, the paper identifies a reusable design rule: spend quantization bits on direction, and build codebooks from the known Gaussian geometry rather than from data clustering.","feed_headline":"Splitting length from direction lifts 2-bit LLM accuracy by 1.5%","feed_subtitle":"A polar-coordinate vector quantizer gives LLM weights separate direction and magnitude codebooks, and zero-shot scores rise.","key_machinery":"The polar-coordinate decoupling transforms each 8-dimensional weight vector into a unit direction and a scalar radius, quantizing the direction by cosine similarity and the radius by scalar quantizer search. The direction codebook is built by greedily sampling directions from the E8 lattice, which the paper uses because E8 gives the densest sphere packing in dimension 8 and its directions are nearly uniform in space. The magnitude codebook is built with the Lloyd-Max algorithm using the probability density function of the root chi-square distribution with 8 degrees of freedom, which is the exact magnitude distribution of a standard Gaussian vector. A randomized Hadamard transform plus per-column scaling is applied first so that every weight column is treated as approximately standard Gaussian, making the two global codebooks reusable across models and layers.","core_discovery":"The paper's central claim is that when weights of a large language model are vector-quantized at 2 bits, the direction of each weight vector carries far more information than its magnitude. On LLaMA-2-7B, quantizing directions alone costs roughly 46.5% zero-shot accuracy, whereas quantizing magnitudes alone costs only 2.3%, and the gap widens as the number of cluster centers shrinks. The authors further argue that the Euclidean distance used by existing vector quantizers over-weights magnitude error, because the magnitude gap enters the squared error quadratically while the direction gap enters approximately linearly. PCDVQ therefore splits each vector into a unit direction and a scalar magnitude, quantizes them with separate codebooks, and allocates more index bits to direction; across six LLMs and five zero-shot tasks, it reports average accuracy gains of at least 1.5% over prior 2-bit methods.","pith_inferences":["A natural extension is to test whether the same direction-over-magnitude sensitivity holds in polar-coordinate KV-cache quantizers, whose phase and radius components could be given asymmetric bit budgets.","The fixed global codebooks rest on the standard-Gaussian premise, so a testable variant is to learn per-layer or per-head codebooks from actual rotated weights; if residual structure survives the Hadamard transform, such data-driven codebooks should outperform the fixed ones.","The decoupling idea could transfer to non-LLM vector quantization workloads with anisotropic data, where E8's spatial uniformity may be less helpful than data-dependent direction codebooks.","The reported throughput gain suggests additional engineering room in fused dequantization kernels that reconstruct vectors as a direction-magnitude product in registers, a possibility the paper measures but does not optimize."],"forward_implications":["Allocating most of a 2-bit budget to a 14-bit direction index and only 2 bits to magnitude is sufficient to beat coupled vector quantization at equal or higher bits on LLaMA-2, LLaMA-3, and Mistral models.","The same precomputed codebooks can be reused for different LLMs because the Hadamard-plus-scaling regularization standardizes weights to a common Gaussian distribution, avoiding per-model codebook training.","Direction error dominates the total quantization error for high-dimensional weight vectors, so vector quantizers that minimize Euclidean distance on coupled vectors are systematically optimizing the wrong quantity.","Adding a single extra bit to the direction codebook at 2.125 bits yields substantial accuracy gains, indicating that the marginal quantization bit is best spent on direction until saturation.","PCDVQ inherits the inference speedups of low-bit weight-only quantization, reporting roughly a tripling of generation throughput, because the compressed weights reduce memory bandwidth."],"supporting_citations":[{"why":"Supplies the argument that a random Hadamard transform makes weight columns approximately Gaussian, which is the premise for the shared global codebooks.","marker":"[5]"},{"why":"Baseline QuIP# method and the source of the Hadamard incoherence processing and fine-tuning procedures that PCDVQ reuses.","marker":"[30]"},{"why":"Establishes E8 as the densest sphere packing in dimension 8, which motivates the spatially uniform direction codebook.","marker":"[33]"},{"why":"Provides the Lloyd-Max algorithm used to construct the optimal scalar magnitude codebook from the root chi-square distribution.","marker":"[21]"},{"why":"Baseline VPTQ method using K-Means vector quantization, whose setup and model are used for the direction-versus-magnitude sensitivity experiment.","marker":"[19]"},{"why":"Supplies the LLaMA-2 models used in the motivating experiment and the main 2-bit evaluations.","marker":"[29]"},{"why":"Supplies the LLaMA-3 models used to test whether the claimed improvement transfers beyond LLaMA-2.","marker":"[11]"}],"fun_headline_variants":["Splitting direction from magnitude adds 1.5% to 2-bit LLM accuracy","Polar coordinate decoupling lifts 2-bit LLM scores by 1.5%","At 2 bits, direction dominates — PCDVQ gains 1.5% on zero-shot","Separate codebooks for length and direction boost LLM quantization","PCDVQ: 2-bit vector quantization with 1.5% higher zero-shot accuracy"],"cache_read_input_tokens":17664,"weakest_assumption_plain":"All claimed gains rest on the premise that after a random Hadamard rotation and per-column scaling, every weight column is close enough to a standard Gaussian distribution that fixed codebooks built from that model are well calibrated for every layer and model.","fun_headline_variants_meta":{"raw":{"variants":["Splitting direction from magnitude adds 1.5% to 2-bit LLM accuracy","Polar coordinate decoupling lifts 2-bit LLM scores by 1.5%","At 2 bits, direction dominates — PCDVQ gains 1.5% on zero-shot","Separate codebooks for length and direction boost LLM quantization","PCDVQ: 2-bit vector quantization with 1.5% higher zero-shot accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000342,"raw_usage":{"total_tokens":1935,"prompt_tokens":1052,"completion_tokens":883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":668,"completion_tokens_details":{"reasoning_tokens":767}},"tokens_in":668,"tokens_out":883,"duration_ms":8945,"temperature":1.0,"reasoning_tokens":767,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:35:44.978183+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a 2-bit PCDVQ run on LLaMA-2-7B, record the actual magnitudes of rotated weight vectors, and compare their quantiles with the root chi-square cumulative distribution used to build the Lloyd-Max codebook; a systematic mismatch would show that the standard-Gaussian premise fails. Equally decisive: replace the global E8 and Lloyd-Max codebooks with per-layer cluster centers at the same bit budget and check whether the reported 1.5% advantage survives.","supporting_citations":[{"cited_title":"The sphere packing problem in dimension 8.Annals of mathematics, pages 991–1015, 2017","cited_arxiv_id":null,"evidence_quote":"Establishes E8 as the densest sphere packing in dimension 8, which motivates the spatially uniform direction codebook."}],"review_version":1}