{"id":"29bb519a-86d5-4c99-b4c9-76acdbb15b1a","arxiv_id":"2506.09932","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A quantization transform combining dynamic per-token mean centering, channel scaling, and Hadamard transforms reduces 4-bit activation quantization error in diffusion transformers, achieving a CLIP score of 31.69 on PixArt-Sigma.","lead":"Diffusion image generation models can be compressed to 4-bit weights and activations with almost no quality loss by adding a simple per-token centering step to existing Hadamard transformations. The method, called HadaNorm, matches the full-precision CLIP score on PixArt-Sigma while using far less memory and compute.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix A W6A6 results contradict the central claim: HadaNorm has highest SQNR but lowest CLIP score, so SQNR is not a valid proxy for generation quality.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: SQNR improvements do not translate to CLIP score improvements, as demonstrated by the paper's own W6A6 results. This is not an external critique but an internal inconsistency in the reported evidence. The algebraic derivation of HadaNorm appears exact, and the W4A4 CLIP score is promising, which is why the paper remains conditionally acceptable rather than rejected. However, without error bars, seeds, and a resolution of the W6A6 discrepancy, the headline claim that HadaNorm 'consistently' reduces quantization error and outperforms SOTA is not established. The recommended verdict remains CONDITIONAL, matching the reader's original verdict, with the condition that the authors provide multi-seed evaluation and either explain the W6A6 SQNR-CLIP divergence or adjust their claims accordingly. The undisclosed alpha value in Eq. (3) should also be reported to rule out tuning on the evaluation metric. These additions would settle whether the central claim holds.","tokens_in":7620,"tokens_out":8844,"duration_ms":98815,"concrete_test":"Run the W6A6 and W4A4 evaluations with at least 5 random seeds on the same COCO subset with 20 denoising steps; report mean and 95% CI for CLIP score and CLIP IQA for each transform. Compute the Spearman correlation between end-to-end SQNR and CLIP across all transforms and bitwidths. If at W6A6 HadaNorm's CLIP remains significantly below No Transform's while its SQNR remains highest, or if the SQNR-CLIP correlation is not positive, then SQNR is not a valid proxy and the claim of consistent improvement is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that HadaNorm 'consistently reduces quantization error across the various components of transformer blocks, outperforming state-of-the-art methods.' This rests on using SQNR as a proxy for generation quality. Appendix A, Table 2 shows this proxy fails within the paper's own results: at W6A6, HadaNorm attains the highest SQNR (3.05 dB) but the lowest CLIP score among quantized methods (31.82), below even No Transform (32.39) and below the full-precision reference's relative ranking. If higher SQNR does not translate to better generation quality, the per-component SQNR improvements in Figures 2 and 4, and the W4A4 headline CLIP gain, are not credible evidence for the method. The absence of error bars or seeds makes the W4A4 CLIP advantage (31.69 vs 31.17 for SDCB) indistinguishable from evaluation noise. Additionally, Eq. (3)'s alpha is tuned on a disjoint calibration set but its value is not reported, so selection on the evaluation metric cannot be excluded.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HadaNorm, a linear post-training quantization transformation for diffusion transformers. The transformation combines per-channel scaling, a Hadamard rotation, and dynamic per-token mean centering before activation quantization, with inverse operations folded into the weights and bias. Equation (1) is an exact identity, so the method does not approximate the layer output. Experiments on PixArt-Sigma at W4A4 report the highest SQNR and CLIP score among the compared transforms, with CLIP 31.69 versus the full-precision 31.66, and improved visual outputs in Figure 3. At W6A6 (Appendix A), however, HadaNorm attains the highest SQNR but its CLIP score is lower than several baselines, which undermines the paper's implicit claim that SQNR improvements translate into better generation quality.","tokens_in":7838,"tokens_out":4692,"duration_ms":54726,"significance":"If the W4A4 result is robust, HadaNorm is a valuable practical contribution: it is an exact, cheap, and simple transformation that enables 4-bit activation quantization of a diffusion transformer with essentially no loss in CLIP score. The derivation in Eq. (1) is exact and parameter-free apart from the calibration-based scale, the inverse transforms are fused into weights, and the dynamic centering adds only one extra token to the sequence. The paper also compares against relevant baselines (SmoothQuant, QuaRot, SDCB). The main weakness is that the empirical support is narrow: one model, one dataset subset, no error bars or multiple seeds, and an internal inconsistency between SQNR and CLIP at W6A6. The significance of the method is therefore conditional on resolving these empirical issues.","major_comments":[{"comment":"The W6A6 results contradict the use of SQNR as a proxy for generation quality. HadaNorm achieves the highest SQNR (3.05 dB) yet its CLIP score (31.82) is below No Transform (32.39), Dynamic Centering (31.87), and QuaRot (31.85), and is essentially tied with SDCB (31.81); CLIP IQA is also flat across all quantized methods while No Transform is highest (0.92). Because Section 4.1 and the abstract present SQNR improvements as evidence that HadaNorm consistently outperforms existing methods, the paper must either explain why the highest SQNR does not yield the best CLIP score at W6A6, or rely on direct generation-quality evidence for the W4A4 claim rather than on SQNR.","section":"Appendix A, Table 2"},{"comment":"All end-to-end results are reported for a single model (PixArt-Sigma), a single COCO subset, and without error bars or multiple seeds. The headline W4A4 CLIP gap (31.69 for HadaNorm vs 31.17 for SDCB) is 0.52 points, which could be evaluation noise; the W6A6 table shows a 0.57-point gap in the opposite direction between No Transform and HadaNorm. Please report standard deviations, bootstrap confidence intervals, or results over at least three seeds, and ideally include a second DiT architecture or dataset to support the generality claim.","section":"Section 4, Table 1"},{"comment":"The exponent alpha in Eq. (3) is tuned on a disjoint calibration set, but its value is never reported and no sensitivity analysis is provided. Since alpha directly controls the channel scaling that is central to HadaNorm, the reader cannot rule out that the empirical gains depend on a particular choice of alpha or on selection of alpha with respect to the evaluation metric. Please report the chosen alpha values (per layer or a summary) and a sensitivity curve showing CLIP/SQNR as a function of alpha.","section":"Section 3.2, Eq. (3)"}],"minor_comments":[{"comment":"The legend entry 'HadaNorm (no scale)' is never defined in the text; please clarify which component is removed relative to full HadaNorm and why this ablation is included.","section":"Figures 2 and 4"},{"comment":"The model name is spelled inconsistently as 'Pixart-Sigma' and 'PixArt-Sigma'; please standardize, and also fix the typo 'litteraure' in Section 4.","section":"Throughout"},{"comment":"The fact that No Transform at W6A6 yields CLIP 32.39, above the full-precision Original 31.66, is unexpected and should be commented on, as it suggests metric noise or prompt/sample variation that may affect the comparisons.","section":"Appendix A, Table 2"},{"comment":"The phrase 'per-token quantization' is used before the dynamic per-token grid is defined; please define the term explicitly or use a different phrase to avoid confusion with per-channel or per-tensor quantization.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The core derivation is sound and the method is elegant, but the empirical grounding needs strengthening. The Appendix A W6A6 SQNR/CLIP mismatch is the most serious issue because it directly undermines the paper's stated evidence. I would not reject the paper, but the authors should either reconcile this inconsistency or reframe the claims around W4A4 generation quality with proper uncertainty quantification. Also note that the evaluation uses only one model and one dataset subset; a second architecture would substantially increase confidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2506.09932. First, the core idea is genuinely simple and new in this literature: take the SmoothQuant-style channel scaling, apply a Hadamard transform, and then subtract a per-token mean computed after the transform. The identity in Eq. (1) is exact, the bias can be absorbed into appended tokens, and the overhead is negligible. The ablation (Fig. 4) shows centering and Hadamard each contribute, and dynamic centering on top of HT gives the best SQNR. That part is solid and worth borrowing.\n\nSecond, the headline W4A4 result (CLIP 31.69 on PixArt-Sigma, matching full precision 31.66) is the main reason to care, but the paper's own Appendix A, Table 2 breaks the argument. At W6A6, HadaNorm has the highest SQNR (3.05 dB) of all transforms and the lowest CLIP score among quantized methods (31.82), below No Transform (32.39). The authors use SQNR gains throughout as evidence of quality, and their own numbers show the proxy fails at the bitwidth where they report it. Maybe the SQNR-CLIP relationship is nonmonotonic, or the W6A6 CLIP differences are noise, but the paper doesn't address that. Combined with no error bars or multiple seeds, the 0.52 CLIP advantage over SDCB at W4A4 is indistinguishable from evaluation noise. Also, alpha in Eq. (3) is tuned on a calibration set and never reported, so we can't know how much selection went into the result.\n\nWhat the paper does well: clear problem statement, an exact derivation, the right baselines cited, and the per-component SQNR analysis is informative. The comparison set is reasonable (SmoothQuant, QuaRot, SDCB). I'd have liked AffineQuant or a static-bias comparison, since this is essentially a dynamic affine per-token correction, but that's a minor omission.\n\nWho should read this: anyone doing PTQ for diffusion transformers or outlier-mitigating rotations. It's a five-minute read and the idea can be plugged into existing frameworks. For a referee: I'd send it out, but with a request for major revision. The authors need to report alpha, add seeds/error bars, and either explain the W6A6 contradiction or reframe the claims so they don't rest on SQNR as a proxy. As is, the evidence supports the method as a heuristic that helps at W4A4, not as a demonstrated SOTA.","headline":"Simple exact centering trick that improves W4A4 quantization of DiTs, but the paper's own W6A6 appendix undermines its SQNR-as-proxy story and the eval is too thin to trust the headline number.","tokens_in":8368,"tokens_out":2611,"would_cite":true,"duration_ms":28625,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that HadaNorm, a mean-centering extension of channel scaling and the Hadamard transform, makes 4-bit diffusion transformer quantization match full precision.","keywords":["Post-training quantization","diffusion transformers","Hadamard transform","activation quantization","channel scaling","outlier mitigation","PixArt-Sigma","CLIP score"],"falsifier":"A held-out evaluation of W4A4 PixArt-Sigma on all COCO 2024 captions with several noise seeds would falsify the empirical claim if HadaNorm's CLIP score falls below SDCB's 31.17 or if its SQNR advantage does not translate into a CLIP advantage. A sharper check already exists inside the paper: at W6A6, HadaNorm has the highest SQNR (3.05 dB) but the lowest CLIP score (31.82) among quantized methods, so reproducing that inversion at 4 bits would directly contradict the mechanism.","tokens_in":7438,"feed_emoji":"⚙️","tokens_out":9984,"duration_ms":101006,"temperature":0.7,"pith_summary":"Post-training quantization can shrink diffusion models, but 4-bit activations usually destroy image quality because of outlier channels. This paper claims that a simple linear preprocessing step — computing each token's mean after scaling its channels and applying a Hadamard transform, then subtracting that mean before quantization — removes enough outliers to make 4-bit weights and activations viable. The paper reports a CLIP score of 31.69 on PixArt-Sigma at W4A4, essentially matching the full-precision 31.66, and shows the extra centering step consistently lowers quantization error across transformer components. If correct, this offers a cheap, calibration-light way to deploy image-generation transformers on memory-constrained devices.","feed_headline":"HadaNorm quantizes PixArt-Sigma to 4 bits without quality loss","feed_subtitle":"Per-token mean subtraction lets 4-bit diffusion transformers match a full-precision CLIP score of 31.66.","key_machinery":"The load-bearing object is the identity in Equation (1): with input $X$, weight $W$, bias $b$, channel scales $\\sigma$, and Hadamard matrix $H$, the paper rewrites $XW + b$ as $(\\tilde{X})(\\tilde{W}) + \\tilde{b}$, where $\\tilde{X} = X \\,\\mathrm{diag}(\\sigma^{-1})H - \\mu$, $\\tilde{W} = H^\\top \\mathrm{diag}(\\sigma)W$, and $\\tilde{b} = b + \\mu \\tilde{W}$. Here $\\mu$ is the per-token mean of the scaled-and-rotated activations, computed dynamically; $\\sigma$ is fixed from calibration as $\\sigma_i = \\max(|X_i|)^\\alpha / \\max(|W_i|)^{1-\\alpha}$. The Hadamard transform spreads outlier energy across channels, the channel scaling reduces scale mismatches, and the dynamic centering removes the mean differences that remain after mixing. This makes both activations and weights more quantizable while leaving the mathematical function unchanged.","core_discovery":"The paper's central discovery is that a Hadamard transform, although effective at mixing channels to reduce heavy-tailed outliers, leaves channels with substantially different means, which wastes a per-token quantization grid. HadaNorm fixes this by first rescaling each channel with static calibration-based scales, applying the Hadamard transform, and then dynamically subtracting the per-token mean of the transformed vector. Because the whole operation is linear, the layer output can be preserved exactly: the inverse scaling and rotation are absorbed into the weights and the centering is compensated by an added bias term $b + \\mu \\tilde{W}$. On PixArt-Sigma, this raises W4A4 quantization to a CLIP score of 31.69, matching the full-precision 31.66, and it improves SQNR on nearly every quantized activation in the transformer, with the textual-token quantizer as the lone exception.","pith_inferences":["The authors leave implicit that the centering step's benefit depends on how much channel means differ; in architectures where channels are already zero-mean, HadaNorm should reduce to a scaled Hadamard transform, so a useful test is to measure per-channel mean spread before deciding where to insert it.","The paper's own W6A6 results show SQNR and CLIP score moving in opposite directions (HadaNorm has the highest SQNR but the lowest CLIP score among quantized methods), so a fair evaluation of the method should report both quantities rather than SQNR alone.","Because the transformation is linear and output-preserving, it can likely be composed with other quantization tools, such as low-rank outlier absorption or time-dependent transforms, to reach 3-bit weights or to cover video diffusion transformers without retraining."],"forward_implications":["PixArt-Sigma can be run with 4-bit weights and activations at a CLIP score of 31.69, slightly above the full-precision 31.66, so no fine-tuning is needed for this aggressive compression level.","The per-token mean is computed on the fly and the inverse of the transform is folded into the weights, so the extra inference cost is just one added token and a bias correction.","The largest SQNR gains appear in the image activations feeding attention and feed-forward blocks and in the output-projection layer, indicating where outlier reduction matters most.","Centering and the Hadamard transform are complementary: the paper's per-component analysis shows the combination, not either transform alone, accounts for the improvement."],"supporting_citations":[{"why":"Introduces per-channel activation scaling (SmoothQuant), the foundation that HadaNorm extends with mean-centering.","marker":"Xiao et al. (2023)"},{"why":"Introduces Hadamard-transform rotation (QuaRot) to remove activation outliers; a direct baseline in the comparison.","marker":"Ashkboos et al. (2024)"},{"why":"Proposes SDCB/Vidit-Q combining channel scaling and Hadamard transform for diffusion transformers; the main baseline HadaNorm improves upon.","marker":"Zhao et al. (2025)"},{"why":"Provides PixArt-Sigma, the diffusion transformer used for all experiments.","marker":"Chen et al. (2024)"},{"why":"Defines CLIP score, the primary metric for generated-image quality throughout the evaluation.","marker":"Hessel et al. (2021)"},{"why":"Provides the COCO 2024 caption subset used for calibration and evaluation.","marker":"Lin et al. (2014)"},{"why":"Provides CLIP IQA, the secondary visual-quality metric reported alongside CLIP score.","marker":"Wang et al. (2023)"}],"fun_headline_variants":["HadaNorm achieves 4-bit diffusion without quality loss","Mean-centered Hadamard transforms enable 4-bit diffusion quantization","HadaNorm: per-token mean subtraction makes 4-bit PixArt match full precision","Diffusion transformer quantization: HadaNorm matches full precision at 4 bits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that reducing quantization noise in the latent space (higher SQNR) is a reliable proxy for better generated images; the paper's own 6-bit results show a case where higher SQNR coincides with a lower CLIP score.","fun_headline_variants_meta":{"raw":{"variants":["HadaNorm achieves 4-bit diffusion without quality loss","Mean-centered Hadamard transforms enable 4-bit diffusion quantization","HadaNorm: per-token mean subtraction makes 4-bit PixArt match full precision","Diffusion transformer quantization: HadaNorm matches full precision at 4 bits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000703,"raw_usage":{"total_tokens":3120,"prompt_tokens":843,"completion_tokens":2277,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":2198}},"tokens_in":459,"tokens_out":2277,"duration_ms":19461,"temperature":1.0,"reasoning_tokens":2198,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:37:02.192821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A held-out evaluation of W4A4 PixArt-Sigma on all COCO 2024 captions with several noise seeds would falsify the empirical claim if HadaNorm's CLIP score falls below SDCB's 31.17 or if its SQNR advantage does not translate into a CLIP advantage. A sharper check already exists inside the paper: at W6A6, HadaNorm has the highest SQNR (3.05 dB) but the lowest CLIP score (31.82) among quantized methods, so reproducing that inversion at 4 bits would directly contradict the mechanism.","supporting_citations":[{"cited_title":"Smoothquant: Accurate and efficient post-training quantization for large language models","cited_arxiv_id":null,"evidence_quote":"Introduces per-channel activation scaling (SmoothQuant), the foundation that HadaNorm extends with mean-centering."},{"cited_title":"L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J","cited_arxiv_id":null,"evidence_quote":"Introduces Hadamard-transform rotation (QuaRot) to remove activation outliers; a direct baseline in the comparison."},{"cited_title":"Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation","cited_arxiv_id":null,"evidence_quote":"Proposes SDCB/Vidit-Q combining channel scaling and Hadamard transform for diffusion transformers; the main baseline HadaNorm improves upon."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the COCO 2024 caption subset used for calibration and evaluation."}],"review_version":1}