{"id":"d41e9a66-9f95-491c-8e44-5ba7978211e4","arxiv_id":"2505.03810","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Sequency-ordering Hadamard rows and grouping them into diagonal blocks reduces 2-bit quantization error, giving training-free perplexity of 11.59 versus 10.97 for learned OSTQuant on Llama-2-7B.","lead":"The paper describes a training-free tweak to rotation-based 2-bit LLM quantization: it reorders the rows of Hadamard matrices by frequency, then applies them in small blocks. On Llama-2-7B it reports WikiText-2 perplexity close to optimization-based methods, without the training cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sequency-ordering benefit is confounded with dropping Hadamard randomization; missing control cells leave the central mechanism untested.","rationale":"The reader's weakest assumption was that Section 3.2's mechanism is asserted rather than derived. I agree partially; the more specific problem is that the experimental design cannot even demonstrate the asserted mechanism because randomization status is confounded with sequency ordering. This is not an outside-consensus objection: the paper's own implementation note creates the confound. The empirical PPL numbers might still hold, and if the control cells confirm sequency ordering, the method is a useful training-free contribution. But as written, the central explanation is unsupported, and the 'free' comparison to learned rotations is weakened by the lack of code and single-model evidence. The verdict should remain conditional: accept only after the control experiments and at least one additional model family are reported.","tokens_in":8876,"tokens_out":14259,"duration_ms":159734,"concrete_test":"On Llama-2-7B W2A16, run QuaRot with R1 set to: (1) randomized Hadamard (existing GH/LH), (2) unrandomized Hadamard, (3) randomized Walsh, i.e., apply the same random diagonal sign flips used for GH/LH to the Walsh matrix, and (4) unrandomized Walsh (existing GW/GSR). Report WikiText-2 PPL for global and local versions. If (2)≈(4), the improvement is from removing randomization; if (3)≈(4), sequency is the active ingredient. Then repeat on Llama-2-13B and Llama-3-8B to check generality beyond one architecture.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that sequency ordering—not block locality or the randomization protocol—reduces 2-bit quantization error. The paper never isolates that variable. Appendix A.1 says 'All Hadamard matrices are randomized... When constructing Walsh matrices, the original Hadamard matrix is used.' A Walsh matrix is a row permutation of the Hadamard matrix; a randomized Hadamard is not. Therefore each Walsh-versus-Hadamard comparison in Table 2 changes two things at once: row/sequency order and the presence of random sign flips. Section 3.2's claim that randomization 'keeps the overall sequency arrangement with no significant changes' is not a proof; random sign flips alter the actual filters and are precisely the incoherence mechanism used by QuIP#. The missing control cells are unrandomized Hadamard and randomized Walsh. For the local case, the rotation block size equals the quantization group size G, so a within-block row permutation cannot change the element multiset of any quantization group under a symmetric group quantizer; the non-permutation difference between GSR and LH is exactly the randomization. Unless GPTQ's order-dependent updates happen to interact with the permutation—an analysis the paper does not provide—the GSR-vs-LH gap cannot be attributed to sequency order. Observation #1 is a matrix-product identity, not a bound on quantization error.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Grouped Sequency-arranged Rotation (GSR), a training-free modification of the Hadamard rotations used in LLM post-training quantization. GSR replaces the standard Hadamard matrix with a sequency-ordered Walsh matrix, optionally in block-diagonal (local) form with block size equal to the quantization group size. The authors evaluate GSR on Llama-2-7B at W2A16 and W2A4 within QuaRot, SpinQuant, and OSTQuant pipelines, reporting improved WikiText-2 perplexity and zero-shot accuracy compared to the baseline rotations, and claiming performance comparable to optimization-based OSTQuant without training.","tokens_in":9120,"tokens_out":9981,"duration_ms":111411,"significance":"If the empirical claims hold, GSR is a practically attractive contribution: it has no learned parameters, is a drop-in replacement for the R1 rotation, and it also improves learned-rotation pipelines such as SpinQuant and OSTQuant. The evaluation is consistent across three quantization pipelines and includes ablations, which is a strength. However, the causal claim that sequency ordering is responsible for the improvement is not currently established, because the Hadamard baselines are randomized while the Walsh variants are not. The theoretical justification in Section 3.2 is an intuition rather than a derivation. The contribution may be salvageable with additional control experiments, but the central mechanism needs to be tested directly.","major_comments":[{"comment":"The comparisons labeled as isolating the sequency arrangement are confounded with randomization. Appendix A.1 states that all Hadamard matrices are randomized while Walsh matrices use the original Hadamard matrix. Therefore each Hadamard-versus-Walsh comparison in Table 2, including GH vs. GW and LH vs. GSR, changes two things at once: the row sequency order and the presence of random sign flips. Random sign flips are not a negligible detail; they are the incoherence mechanism used by QuIP# and QuaRot. The statement in Section 3.2 that randomization keeps the sequency arrangement 'with no significant changes' is not a proof. To support the paper's central claim, the authors should add the missing control cells: unrandomized global Hadamard, randomized global Walsh, unrandomized local Hadamard, and randomized local Walsh.","section":"Appendix A.1, Table 2"},{"comment":"Observation #1 is a matrix-product identity, not a derivation that sequency ordering reduces quantization error. The text moves from 'smaller sequency variance within each column group' to 'the Walsh matrix will produce rotated weight groups with fewer massive outliers' without a formal argument connecting sequency variance to the quantization error of a group quantizer (RTN or GPTQ). A concrete bound, or at least a variance calculation on the entries of the rotated weight groups, is needed before the title's claim that the transformation is 'optimized' for quantization can be accepted. Alternatively, the paper should explicitly reframe Section 3.2 as an empirical hypothesis rather than a justification.","section":"Section 3.2, Eq. (4) and Observation #1"},{"comment":"The claim that 'For Rr, the arrangement has no impact as long as the set of sequency values is equal' is not justified and appears inconsistent with Eq. (4). Since W' = Rf^{-1} W Rr, the row arrangement of Rr changes which linear combination of columns of the intermediate matrix is taken in each row of W'. Unless the quantizer is invariant under the resulting column transformation, the element multiset of a row quantization group can change. The authors should either prove the invariance for group quantization or run a control that swaps only the rear rotation Rr between Hadamard and Walsh order.","section":"Section 3.2, paragraph after Observation #1"},{"comment":"The zero-shot averages are not directly comparable across methods because they are computed over different task sets. QuaRot and SpinQuant use eight tasks including both LAMBADA variants, while OSTQuant uses nine tasks including BoolQ, OpenBookQA, and SIQA. The headline claim that GSR is 'comparable' to OSTQuant is therefore mainly supported by the WikiText-2 PPL column. The authors should either report per-overlap-task averages for cross-method comparisons or clearly mark that the zero-shot columns refer to different task sets.","section":"Table 2, zero-shot columns"}],"minor_comments":[{"comment":"There is a typo: 'Walsh matirx' should be 'Walsh matrix'.","section":"Section 4"},{"comment":"The Walsh matrix is written as an entrywise ±1 matrix, but for it to be a rotation matrix it must be scaled by 1/sqrt(G). This normalization should be stated explicitly in the definition of Hwal.","section":"Eq. (3)"},{"comment":"The description that randomization 'flips the signs of diagonal elements in a Hadamard matrix' is imprecise. The randomized Hadamard transform multiplies the Hadamard matrix by a random diagonal sign matrix, i.e., H D or D H, which is not the same as flipping only the diagonal entries of H. Since this distinction is directly relevant to the missing control cells, the wording should be corrected.","section":"Section 3.2, comparing RHT and Walsh"},{"comment":"Several references have formatting issues: 'Yonatan Bisk, ..., and 1 others', 'Hugo Touvron, ..., and 1 others', and 'Le Dinh Chon Tam and R.Y . Goulet' have spacing/author-list problems. These should be cleaned up.","section":"References"},{"comment":"The ablation on R4 mentions that local rotation disables the fast Hadamard transform and requires storing an FP32 tensor during inference. Since GSR is applied to R1 in the main experiments, clarify whether the same inference-time overhead applies to R1 or whether R1 is fully fused into the weights offline.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The empirical recipe may be useful, but the paper currently presents a causal mechanism that the experiments do not isolate. I would like the editor to weigh whether the missing control cells can be added within a revision cycle; if the authors cannot run them, they should substantially weaken the mechanism claims and present GSR as an empirical, training-free rotation choice. I also noted that no code release is mentioned, which is increasingly expected for quantized-LLM papers reporting benchmark numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my read of the GSR paper. The headline result: swapping QuaRot's randomized Hadamard for a Walsh (sequency-ordered) matrix, applied block-wise, gives a real jump at W2A16/W2A4 on Llama-2-7B—PPL 11.59 vs 20.29 for QuaRot, near OSTQuant's 10.97 without training. That's an interesting empirical trick, and the paper shows it also helps as a warm start for SpinQuant and OSTQuant. Good things: the experiments are consistent across three pipelines, the ablations are honest (they flag the R4 inference cost), and the paper is clearly written. Counting the method as new is fair: using sequency order for rotation in quantization isn't in the cited literature.\n\nThe soft spot is the central mechanism claim. The comparison between GH (randomized Hadamard) and GW (original Walsh) changes two variables at once: the row ordering and the presence of the random sign flips that QuIP# and QuaRot use for incoherence. Same for LH vs GSR. The paper asserts that randomization 'keeps the overall sequency arrangement with no significant changes,' but flipping signs in a Hadamard does change the sequency of each row, and more importantly it changes the actual rotation. The missing control cells—unrandomized Hadamard and randomized Walsh—are exactly the ones needed to tell whether the gain comes from sequency alignment or from dropping randomization. The theoretical justification in Section 3.2 is an intuition, not a derivation, and Observation #1 is just a matrix-product identity. For the local case, since block size equals the quantization group size, a within-block row permutation alone can't change the element multiset in each group; GPTQ's order-dependent updates might interact with the permutation, but the paper doesn't analyze that.\n\nSo the method may be right, but the reason the paper gives for why it works is untested. That's a load-bearing flaw in the story, though not necessarily in the method. The evidence is one model, no code, and the practical claim about R4 is left unresolved. Still, the empirical pattern is clear enough that the work deserves a serious referee. My call: peer review yes, but with a requirement for the missing control experiments and at least one more model. It's a useful contribution to the quantization subfield if the mechanism gets sorted out.","headline":"The method looks genuinely useful at 2-bit, but the paper's own numbers don't isolate the sequency ordering it claims is load-bearing.","tokens_in":9641,"tokens_out":4598,"would_cite":false,"duration_ms":48539,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reordering a Hadamard matrix by sequency and applying it in blocks makes training-free 2-bit LLM quantization competitive with optimized methods.","keywords":["post-training quantization","LLM quantization","rotation transformation","Walsh-Hadamard transform","sequency ordering","2-bit weights","group quantization","outlier mitigation"],"falsifier":"Take a weight matrix from Llama-2-7B, rotate it separately with the global Hadamard and the global Walsh matrix, and measure the maximum absolute value within each quantization group before quantizing. If the Walsh-rotated groups do not have smaller within-group outlier magnitudes, or if the per-group quantization error is not lower, the stated sequency mechanism is falsified.","tokens_in":8694,"feed_emoji":"🔄","tokens_out":5148,"duration_ms":50957,"temperature":0.7,"pith_summary":"The paper claims that a zero-training change to how LLM weights are rotated before quantization can match, and in some settings beat, optimization-based 2-bit quantization methods. Its central move is to replace the standard randomized Hadamard rotation with a Walsh matrix—the same Hadamard matrix with rows reordered by sequency, the number of sign changes per row—and to apply it in blocks aligned with the quantization group. On Llama-2-7B with 2-bit weights and 16-bit activations, this Grouped Sequency-arranged Rotation (GSR) lowers WikiText-2 perplexity from 20.29 with the standard rotation to 11.59, within 0.62 of the trained OSTQuant's 10.97, while raising average zero-shot accuracy from 32.06 percent to 42.44 percent. Under 2-bit weights and 4-bit activations, GSR gives perplexity 15.23, better than OSTQuant's 16.16. If it holds, this matters because 2-bit quantization would roughly quarter the memory footprint of deployed models without requiring the optimization phase that current low-bit methods need.","feed_headline":"Grouped Walsh rotation matches trained 2-bit LLM quantization","feed_subtitle":"Training-free reordering of a Hadamard matrix cuts WikiText-2 perplexity from 20.29 to 11.59 on Llama-2-7B.","key_machinery":"The central object is the Walsh matrix: a Hadamard matrix with rows permuted by bit-reversal and Gray-code so that the sequency of each row, the number of sign flips, is in ascending order. Sequency is treated as the discrete analogue of frequency, so filters of similar sequency produce rotated weight groups with smaller within-group outlier spread. GSR combines this ordering with a block-diagonal rotation whose block size equals the quantization group size, so each quantization group is rotated independently and remaining outliers stay inside one group. A supporting identity, Observation #1, shows that under group quantization each row group of the rotated weight is generated by the corresponding group of filters in the front rotation matrix and by all filters of the rear matrix, which is what lets the paper attribute the improvement to the arrangement of sequency filters.","core_discovery":"The central discovery is that the ordering of rows inside a rotation matrix matters for group quantization, not just the rotation as a whole. Replacing the natural-order Hadamard matrix used in QuaRot with a Walsh matrix—the same entries permuted so that sequency increases linearly—reduces within-group variance of the rotated weights, and applying this Walsh rotation as a block-diagonal matrix whose block size matches the quantization group confines outliers to a single group. The paper shows that on Llama-2-7B, this training-free GSR achieves WikiText-2 PPL 11.59 and 42.44 percent average zero-shot accuracy at W2A16, compared with 20.29 and 32.06 percent for the standard global randomized Hadamard rotation, and that at W2A4 it reaches PPL 15.23, lower than the optimization-based OSTQuant's 16.16. The same change also improves SpinQuant and OSTQuant when it replaces their rotation, for example cutting OSTQuant's W2 PPL from 10.97 to 9.03.","pith_inferences":["The paper does not test whether the benefit persists if the block size of the rotation and the quantization group size are decoupled; if the mechanism is right, misaligning them should create a sharp performance drop, which would be a direct test of the confinement claim.","The authors note in the appendix that local rotation on R4 disables the fast Hadamard transform and requires storing the full matrix in memory; this practical cost is not reflected in the headline perplexity numbers.","Sequency ordering is one specific permutation of Hadamard rows, and the paper does not explore other permutations that might further reduce within-group outliers for particular weight distributions.","The improvement is claimed only for extremely low-bit group quantization; the paper's own limitation statement says that at larger bit-widths the quantization error becomes small enough that sequency alignment shows no visible gain."],"forward_implications":["Training-free 2-bit weight quantization of Llama-2-7B reaches WikiText-2 PPL 11.59, within 0.62 of the optimization-based OSTQuant's 10.97, while skipping the training phase entirely.","Under W2A4, GSR applied to QuaRot reaches PPL 15.23, outperforming OSTQuant's 16.16 with fewer resources.","GSR also strengthens learned-rotation methods when used as their rotation: OSTQuant with GSR improves W2 PPL from 10.97 to 9.03 and zero-shot accuracy from 45.52 percent to 50.51 percent.","The sequency reordering alone, without local blocking, improves QuaRot's W2 PPL from 20.29 to 15.38, showing that row ordering and block structure make separable contributions."],"supporting_citations":[{"why":"Supplies the QuaRot baseline, the randomized Hadamard rotation on R1 that GSR replaces, and the quantization setup used for comparison.","marker":"Ashkboos et al. 2024"},{"why":"Provides the learned-rotation SpinQuant baseline and the terminology for rotation placement; GSR is tested as an improved initialization for it.","marker":"Liu et al. 2025"},{"why":"Defines the optimization-based OSTQuant baseline with PPL 10.97 that GSR approaches training-free, and one of the methods GSR further improves.","marker":"Hu et al. 2025"},{"why":"Introduces the local block-diagonal rotation idea that GSR combines with Walsh sequency ordering.","marker":"Lin et al. 2024"},{"why":"Explains the randomized Hadamard transform and sign-flip randomization used in QuaRot, which the paper compares against the Walsh sequency arrangement.","marker":"Tseng et al. 2024"},{"why":"Provides the formal construction of the Walsh matrix via bit-reversal and Gray-code permutations, which defines the sequency ordering used by GSR.","marker":"Tam and Goulet 1972"}],"fun_headline_variants":["Training-free grouped Walsh rotation outperforms tuned 2-bit quant","Walsh reorder cuts 2-bit LLM perplexity from 20.29 to 11.59","Free rotation trick matches trained 2-bit LLM quantization","Grouped sequency rotation: 2-bit quant boost without training","Block-diagonal Walsh rotation improves 2-bit LLM quantization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument stands on the claim, stated rather than derived, that sequency behaves like frequency in filtering, so that sorting rotation filters by sequency produces rotated weight groups with fewer massive outliers.","fun_headline_variants_meta":{"raw":{"variants":["Training-free grouped Walsh rotation outperforms tuned 2-bit quant","Walsh reorder cuts 2-bit LLM perplexity from 20.29 to 11.59","Free rotation trick matches trained 2-bit LLM quantization","Grouped sequency rotation: 2-bit quant boost without training","Block-diagonal Walsh rotation improves 2-bit LLM quantization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000639,"raw_usage":{"total_tokens":2942,"prompt_tokens":943,"completion_tokens":1999,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":1902}},"tokens_in":559,"tokens_out":1999,"duration_ms":15065,"temperature":1.0,"reasoning_tokens":1902,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:24:18.009816+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a weight matrix from Llama-2-7B, rotate it separately with the global Hadamard and the global Walsh matrix, and measure the maximum absolute value within each quantization group before quantizing. If the Walsh-rotated groups do not have smaller within-group outlier magnitudes, or if the per-group quantization error is not lower, the stated sequency mechanism is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the local block-diagonal rotation idea that GSR combines with Walsh sequency ordering."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Explains the randomized Hadamard transform and sign-flip randomization used in QuaRot, which the paper compares against the Walsh sequency arrangement."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the formal construction of the Walsh matrix via bit-reversal and Gray-code permutations, which defines the sequency ordering used by GSR."}],"review_version":1}