{"id":"bedb0d9d-b4fc-4ce3-9de1-a1bebddd1e06","arxiv_id":"2605.26339","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"QAM-W introduces joint 2D codebook quantization for LLM weights via Hadamard rotation and activation-aware scaling, achieving near full-precision perplexity at ~5.5 bpw.","lead":"QAM-W is a quantization codec for LLM weights that normalizes rows, applies block-Hadamard rotation, pairs coordinates into 2D points, and quantizes them with a single Lloyd-Max codebook plus activation-aware scaling. It reports near-BF16 perplexity at roughly 5.5 bits per weight across multiple models while using fewer bits than 8-bit baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Lloyd-Max codebook optimality after block-Hadamard rotation + activation-aware scaling is assumed without distribution verification","rationale":"The reader's weakest_assumption is precisely the load-bearing point; the empirical cross-model results and KL-PPL correlation provide supporting evidence but do not close the distribution-match gap. No stronger internal inconsistency appears from the given claims.","tokens_in":1815,"tokens_out":343,"duration_ms":15438,"concrete_test":"For one model (e.g., Llama-7B layer 5), extract the 2D coordinates after rotation and activation scaling, compute their empirical 2D density on a 128×128 grid, and measure KL divergence to the unit circular Gaussian; if average KL > 0.05 nats, retrain the codebook on the observed coordinates and re-run WikiText-2 PPL on that model to quantify any degradation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that L2-normalized, block-Hadamard-rotated, paired 2D coordinates remain close enough to the unit circular Gaussian that the fixed Lloyd-Max codebook stays near-optimal, and that activation-aware per-channel scaling does not induce a distribution shift large enough to invalidate the ±0.4% PPL bound. The abstract reports strong empirical results and a high Spearman correlation between paired KL and ΔPPL, but provides no explicit check (histogram, KL, or ablation) that the post-transform marginals match the training distribution; if they diverge, the reported quality at 5.5 bpw could be an artifact of the particular models rather than a property of the codec.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces QAM-W, a post-training weight quantization codec for LLMs that L2-normalizes each row, applies block-Hadamard rotation, pairs coordinates into 2D, and quantizes against a fixed Lloyd-Max codebook trained on the unit circular Gaussian, augmented by activation-aware per-channel scaling. Across five models (1.1B–13B) from four families and eight configurations, the activation-aware variant at ≈5.5 bpw is reported to stay within ±0.4% of BF16 WikiText-2 perplexity, matching SmoothQuant W8A8 quality at 32% fewer bits; joint 2D coding beats polar coding, and paired KL tracks ΔPPL% with Spearman ρ=0.99 over 37 rows. A 3.5 bpw variant is competitive on tolerant architectures, while QTIP is noted as superior at strict 4 bpw.","tokens_in":1984,"tokens_out":708,"duration_ms":22117,"significance":"If the empirical claims hold after verification, the work would demonstrate that recovering pairwise coordinate structure via rotation and 2D coding can preserve LLM quality at 5–6 bpw, offering a practical middle ground between 4 bpw and 8 bpw methods. The reported high Spearman correlation between KL and perplexity change provides suggestive support for a distortion-to-PPL link, and the cross-model, cross-bitwidth scope strengthens the result if protocols are complete. The contribution is localized to the 5–6 bpw regime rather than claiming broad superiority.","major_comments":[{"comment":"Abstract: the central performance claim (activation-aware QAM-W at ≈5.5 bpw stays within ±0.4% of BF16 PPL on every model) rests on the unverified assumption that L2-normalized, block-Hadamard-rotated, activation-scaled 2D coordinates remain sufficiently close to the unit circular Gaussian for the fixed Lloyd-Max codebook to stay near-optimal; no histogram, marginal KL, or ablation confirming post-transform distribution match is referenced, which directly affects whether the reported quality is a property of the codec or an artifact of the tested models.","section":"Abstract"},{"comment":"Abstract: the monotone composite bound from codec distortion to KL to PPL is asserted to explain the Spearman ρ=0.99 but is not derived or proven; without the derivation or an explicit statement that the 37-row correlation was pre-specified, the correlation cannot be treated as confirmatory evidence for the bound.","section":"Abstract"},{"comment":"Abstract / experimental description: no error bars, dataset splits, full training/evaluation protocol, or verification that the 37-row Spearman analysis was pre-registered are supplied, rendering the central cross-model claim impossible to assess for statistical reliability.","section":"Abstract"}],"minor_comments":[{"comment":"Abstract: the exact definition of 'bpw' (including overhead for scaling factors) and the precise pairing strategy after Hadamard rotation should be stated explicitly rather than left implicit.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an empirical methods paper whose soundness hinges on experimental completeness; the absence of distribution checks and protocol details is the primary barrier to evaluation rather than any internal contradiction in the described method."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive feedback. We address each major comment below with clarifications and proposed changes to the manuscript.","responses":[{"response":"We agree that explicit verification of the post-transform distribution would strengthen the central claim. The Hadamard rotation is selected because it is known to approximately Gaussianize and decorrelate coordinates for a wide range of input distributions, and L2 normalization enforces unit scale. To directly address the concern, we will add histograms of the rotated 2D coordinates, marginal KL values to the unit circular Gaussian, and an ablation removing the rotation in the revised manuscript.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central performance claim (activation-aware QAM-W at ≈5.5 bpw stays within ±0.4% of BF16 PPL on every model) rests on the unverified assumption that L2-normalized, block-Hadamard-rotated, activation-scaled 2D coordinates remain sufficiently close to the unit circular Gaussian for the fixed Lloyd-Max codebook to stay near-optimal; no histogram, marginal KL, or ablation confirming post-transform distribution match is referenced, which directly affects whether the reported quality is a property of the codec or an artifact of the tested models."},{"response":"The manuscript states the correlation is 'consistent with' a monotone composite bound rather than claiming a formal derivation or proof. The observed ρ=0.99 provides empirical support for using paired KL as a proxy but does not constitute confirmatory evidence of a bound. We will revise the wording to remove any implication of a proven bound and clarify that the relationship is empirical.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the monotone composite bound from codec distortion to KL to PPL is asserted to explain the Spearman ρ=0.99 but is not derived or proven; without the derivation or an explicit statement that the 37-row correlation was pre-specified, the correlation cannot be treated as confirmatory evidence for the bound."},{"response":"We will expand the experimental section with the complete evaluation protocol, including WikiText-2 splits and quantization procedure. Error bars will be added for main results via repeated runs with different seeds. The 37-row Spearman analysis was exploratory rather than pre-registered; we will state this explicitly.","revision_made":"partial","referee_comment":"[Abstract] Abstract / experimental description: no error bars, dataset splits, full training/evaluation protocol, or verification that the 37-row Spearman analysis was pre-registered are supplied, rendering the central cross-model claim impossible to assess for statistical reliability."}],"tokens_in":1654,"tokens_out":596,"duration_ms":31755,"standing_objections":["Verification that the 37-row Spearman analysis was pre-registered (it was not pre-registered)"]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core contribution is a weight codec that L2-normalizes rows, applies block-Hadamard rotation, pairs the coordinates, and quantizes them with a single Lloyd-Max codebook trained on the unit circular Gaussian, plus an activation-aware per-channel scale. At roughly 5.5 bits per weight the activation-aware version stays inside 0.4 percent of BF16 WikiText-2 perplexity on five models from four families, and it beats polar amplitude-phase coding by a noticeable margin at the same rate.\n\nThe experiments are the strongest part: a cross-model sweep, direct head-to-head with SmoothQuant W8A8 and QTIP, and the reported Spearman correlation of 0.99 between paired KL and perplexity shift. Those numbers give a reader something concrete to check.\n\nThe soft spot is exactly the one the stress-test flags. The method assumes the post-rotation, post-pairing marginals remain close enough to the unit circular Gaussian that the fixed codebook stays near-optimal, and that the activation scaling does not shift the distribution enough to break the reported bound. The abstract states the results but supplies no histogram, no post-transform KL, and no ablation that would confirm the assumption. If that match does not hold, the 5.5 bpw numbers could be tied to the particular models rather than to the codec itself. The paper also gives no error bars and no full protocol for the 37-row correlation, which makes the central claim harder to assess from the text alone.\n\nThis is aimed at people who need practical weight compression for constrained hardware. The method is concrete enough and the empirical scope broad enough that it deserves a serious referee who can ask for the distribution checks and the missing protocol details.","headline":"QAM-W delivers usable 5.5 bpw results on several models but rests on an unverified claim that the rotated weights stay close to the circular Gaussian the codebook was trained on.","tokens_in":2454,"tokens_out":438,"would_cite":false,"duration_ms":15745,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"QAM-W recovers pairwise structure in LLM weight rows via Hadamard rotation and 2D codebook quantization, staying within 0.4 percent of BF16 perplexity at 5.5 bits per weight.","keywords":["LLM weight quantization","2D codebook quantization","Hadamard rotation","activation-aware scaling","post-training quantization","Lloyd-Max codebook","perplexity preservation"],"falsifier":"Quantize one of the tested models at 5.5 bpw with QAM-W, evaluate perplexity on C4 or a downstream task suite, and check whether the deviation from BF16 exceeds 0.4 percent.","tokens_in":2714,"feed_emoji":"","tokens_out":843,"duration_ms":29946,"temperature":0.7,"pith_summary":"The paper introduces QAM-W as a post-training weight quantizer that restores coordinate pairing lost in scalar methods. Rows are L2-normalized, transformed by a block-Hadamard rotation, paired into 2D points, and quantized to a single Lloyd-Max codebook trained on the unit circular Gaussian, then scaled per channel using activation statistics. Across five models from four families the activation-aware version at approximately 5.5 bits per weight keeps WikiText-2 perplexity inside plus or minus 0.4 percent of the BF16 baseline. This performance matches SmoothQuant W8A8 while using 32 percent fewer weight bits, and 2D joint coding beats separate amplitude-phase quantization by 2 to 15 points in relative perplexity change.","feed_headline":"2D codebook keeps LLM perplexity near BF16 at 5.5 bpw","feed_subtitle":"QAM-W matches SmoothQuant W8A8 quality at 32 percent fewer weight bits across five models from four families.","key_machinery":"Block-Hadamard rotation of normalized rows followed by 2D pairing and quantization against a single Lloyd-Max codebook trained on the unit circular Gaussian, combined with activation-aware per-channel scaling.","core_discovery":"QAM-W quantizes LLM weights by L2-normalizing each row, applying a block-Hadamard rotation, pairing the rotated coordinates into 2D points, and mapping those points to a Lloyd-Max codebook trained on the unit circular Gaussian, with an additional activation-aware per-channel scaling factor; the resulting codec keeps perplexity within plus or minus 0.4 percent of BF16 at roughly 5.5 bits per weight across the tested models and configurations.","pith_inferences":["The same rotation-plus-2D-pairing pattern could be tested on activation tensors or on attention matrices to reduce total model size further.","The high KL-to-perplexity correlation suggests KL divergence on a small calibration set could serve as a fast proxy metric for ranking new quantizers.","Applying the method to models larger than 13B or to instruction-tuned checkpoints would test whether the 0.4 percent bound holds outside the current range.","Layer-wise adaptation of the codebook scale or rotation block size might close the remaining gap at 4 bpw."],"forward_implications":["Joint 2D coding outperforms polar amplitude-by-phase coding by 2-15 percentage points in relative perplexity change at equal bitrate.","Paired KL divergence against BF16 tracks relative perplexity change with Spearman correlation 0.99 across 37 method-model combinations.","The activation-aware variant matches SmoothQuant W8A8 quality at 32 percent fewer weight bits.","A 3.5 bpw configuration remains competitive on quantization-tolerant architectures.","At strict 4 bpw the QTIP rotated-codebook method outperforms QAM-W."],"fun_headline_variants":["Hadamard-rotated 2D codebook sustains near BF16 PPL at 5.5 bpw","QAM-W 2D codebook matches BF16 at 5.5 bpw across five LLMs","Paired 2D quantization keeps PPL change under 0.4% at 5.5 bpw","Lloyd-Max 2D codebook with scaling nears BF16 quality at 5.5 bpw"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The Lloyd-Max codebook trained on the unit circular Gaussian remains near-optimal after the block-Hadamard rotation, and activation-aware scaling introduces no distribution shift large enough to invalidate the reported perplexity bounds.","fun_headline_variants_meta":{"raw":{"variants":["Hadamard-rotated 2D codebook sustains near BF16 PPL at 5.5 bpw","QAM-W 2D codebook matches BF16 at 5.5 bpw across five LLMs","Paired 2D quantization keeps PPL change under 0.4% at 5.5 bpw","Lloyd-Max 2D codebook with scaling nears BF16 quality at 5.5 bpw"]},"model":"grok-4.3","cost_usd":0.009466,"raw_usage":{"total_tokens":4270,"prompt_tokens":752,"num_sources_used":0,"completion_tokens":110,"cost_in_usd_ticks":94662000,"prompt_tokens_details":{"text_tokens":752,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3408,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":752,"tokens_out":110,"duration_ms":27075,"temperature":1.0,"reasoning_tokens":3408,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T22:11:31.266090+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Quantize one of the tested models at 5.5 bpw with QAM-W, evaluate perplexity on C4 or a downstream task suite, and check whether the deviation from BF16 exceeds 0.4 percent.","supporting_citations":[],"review_version":1}