{"id":"bde4382c-5fa5-4418-990e-a2eaf4342f48","arxiv_id":"2606.07618","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"ScaleSweep sweeps block scale candidates for NVFP4 quantization to minimize MSE/WMSE, derives bounds to shrink the search, and reports improved performance on Llama and Qwen models over AbsMax initialization.","lead":"The paper introduces ScaleSweep, a method that sweeps possible block scale values for NVFP4 4-bit quantization of LLMs and picks the one minimizing MSE or WMSE error. Smart readers might care because better low-precision quantization could let large models run on cheaper hardware with less accuracy loss.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Per-tensor MSE/WMSE minimization may not align with optimal end-to-end accuracy under joint quantization of weights, activations, KV cache and query states","rationale":"The identified load-bearing assumption matches the reader's weakest_assumption exactly. The abstract-only review produced UNVERDICTED; the same assumption remains the decisive unverified link even after the full manuscript is considered, so the verdict should move to CONDITIONAL pending the proposed check.","tokens_in":1734,"tokens_out":347,"duration_ms":14943,"concrete_test":"On Llama-7B with the same calibration set, replace ScaleSweep's per-tensor MSE selection with a direct search (e.g., 20-point grid or Bayesian optimization) over the same bounded range that minimizes next-token cross-entropy on a held-out validation split; if the resulting end-to-end perplexity or zero-shot accuracy improves by >3 % relative, the per-tensor proxy is insufficient.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on ScaleSweep choosing, for each block, the scale inside the derived bounds that minimizes per-tensor MSE or WMSE. For the reported >93 % retention of full-precision performance to follow, this local proxy must produce scales that remain near-optimal once quantization errors from all four tensor types interact across layers. The theoretical bounds are derived only for the isolated per-tensor reconstruction error; no argument shows they contain the scale that would minimize end-to-end loss. Experiments compare against AbsMax baselines but do not compare against scales obtained by any end-to-end objective, leaving open the possibility that a different per-block choice would close more of the remaining gap.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces ScaleSweep, a post-training method for NVFP4 quantization of LLMs. For each block it sweeps a reduced set of scale candidates (bounded via a theoretical analysis of per-tensor MSE and WMSE) and retains the candidate that minimizes the chosen objective. The bounds are claimed to contain the per-tensor optimum while keeping overhead negligible. Experiments on Llama and Qwen models report consistent gains over AbsMax-style initialization and, under simultaneous quantization of weights, activations, KV cache and query states, retention of more than 93 % of full-precision performance.","tokens_in":1892,"tokens_out":348,"duration_ms":20281,"significance":"If the reported end-to-end gains are reproducible and the per-tensor proxy is shown to be sufficient, the work supplies a practical, low-overhead improvement to a hardware-supported 4-bit format that narrows the accuracy gap for large models. The explicit derivation of sweep bounds is a methodological strength.","major_comments":[{"comment":"Abstract and Experiments: The central performance claim (>93 % retention under joint quantization of four tensor types) rests on the unverified assumption that the scale minimizing isolated per-tensor MSE/WMSE also minimizes (or nearly minimizes) the end-to-end loss once quantization errors from weights, activations, KV cache and query states interact across layers. The bounds and objective are derived only for per-tensor reconstruction; no argument or ablation demonstrates that these locally optimal scales remain near-global optima, and experiments compare solely against AbsMax rather than against any end-to-end objective.","section":"Abstract and Experiments"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback highlighting the distinction between per-tensor and end-to-end objectives. We respond to the major comment below.","responses":[{"response":"We agree that ScaleSweep optimizes a per-tensor reconstruction objective (MSE or WMSE) and that the manuscript provides neither a theoretical argument nor an ablation showing these local choices remain near-optimal once errors interact across layers and tensor types. The bounds are derived strictly for the isolated per-tensor case. An end-to-end objective would require back-propagation through the full model and all quantization operators, which is computationally prohibitive for the LLMs considered and inconsistent with the goal of a negligible-overhead post-training initializer. Prior quantization work has likewise relied on per-tensor proxies for the same practicality reasons. Our experiments demonstrate that the resulting scales, when applied jointly to weights, activations, KV cache and query states, consistently outperform the AbsMax baseline and retain >93 % of full-precision accuracy; this supplies empirical support for the proxy but does not constitute a proof of global optimality. We compare only against AbsMax because it is the de-facto initialization in NVFP4 hardware flows; constructing and evaluating an end-to-end baseline lies outside the scope of the present contribution. In revision we will add a short discussion in the experiments section clarifying the proxy nature of the objective and the empirical nature of the end-to-end claims.","revision_made":"partial","referee_comment":"[Abstract and Experiments] Abstract and Experiments: The central performance claim (>93 % retention under joint quantization of four tensor types) rests on the unverified assumption that the scale minimizing isolated per-tensor MSE/WMSE also minimizes (or nearly minimizes) the end-to-end loss once quantization errors from weights, activations, KV cache and query states interact across layers. The bounds and objective are derived only for per-tensor reconstruction; no argument or ablation demonstrates that these locally optimal scales remain near-global optima, and experiments compare solely against AbsMax rather than against any end-to-end objective."}],"tokens_in":1344,"tokens_out":430,"duration_ms":18518,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper introduces ScaleSweep: instead of using AbsMax to set the per-block scale in NVFP4, it sweeps a small set of candidate scales and keeps the one that minimizes MSE or WMSE on the reconstructed tensor. They also derive lower and upper bounds on the search range that are guaranteed to contain the optimal candidate for that per-tensor objective, which keeps the extra cost low.\n\nThat bounded-sweep idea plus the accompanying analysis is the clearest new piece. It directly tackles the initialization gap the abstract mentions and gives a concrete, cheap way to do better than the baseline.\n\nThe experiments claim consistent gains on Llama and Qwen and, under full end-to-end quantization of the four tensor types, more than 93 % retention of full-precision performance. If those numbers hold up in the full paper, the method is worth trying for anyone already targeting NVFP4 hardware.\n\nThe soft spot is the one flagged in the stress-test note. The scale choice is made to minimize isolated per-tensor reconstruction error, yet the headline result is about model accuracy after all four tensor types interact across layers. No comparison is described against scales that were chosen or adjusted with an end-to-end loss, so it is still open whether a different per-block decision would close more of the remaining gap. The abstract states the bounds preserve the optimal candidate, but that optimality is only with respect to the local MSE/WMSE, not the downstream metric.\n\nThis is a practical, incremental paper aimed at the post-training quantization crowd. The core procedure is straightforward and the bounds look like a useful engineering addition. I would send it to peer review; the method is concrete enough that referees can check the claims and the authors can add the missing end-to-end comparison if needed.","headline":"ScaleSweep adds a bounded sweep over NVFP4 block scales that beats AbsMax on the reported models, but the per-tensor MSE/WMSE objective is not shown to be optimal once weights, activations, KV cache and queries are quantized together.","tokens_in":2352,"tokens_out":452,"would_cite":false,"duration_ms":19663,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"ScaleSweep sweeps bounded block-scale candidates to minimize MSE or WMSE and improve NVFP4 quantization accuracy for LLMs.","keywords":["NVFP4","post-training quantization","large language models","block scales","scale initialization","mean square error","quantization error"],"falsifier":"A measured case in which a scale lying outside the derived bounds yields lower end-to-end model error than every scale inside the bounds.","tokens_in":2634,"feed_emoji":"","tokens_out":685,"duration_ms":19885,"temperature":0.7,"pith_summary":"NVFP4 uses fine-grained block scales to raise the quality of 4-bit floating-point quantization, yet standard AbsMax initialization still leaves a gap to the best possible result. The paper introduces ScaleSweep, which tests a set of candidate scales for each block and keeps the one that produces the smallest reconstruction error under either plain or weighted mean square error. Theoretical lower and upper bounds are derived for the search interval so that the optimal scale is never missed while the number of trials stays small. On Llama and Qwen models the method raises end-to-end accuracy over prior initializers. When weights, activations, KV cache, and query states are all quantized together, more than 93 percent of full-precision performance is retained.","feed_headline":"Bounded scale sweep improves NVFP4 quantization of LLMs","feed_subtitle":"Selecting the MSE-minimizing block scale inside derived bounds narrows the gap to full precision with negligible extra cost.","key_machinery":"ScaleSweep: a sweep over feasible block-scale candidates that selects the value minimizing MSE or WMSE, made efficient by derived lower and upper bounds on the scale range.","core_discovery":"ScaleSweep is a scale optimization method that sweeps feasible block scale candidates and selects the candidate that minimizes a target objective. We further provide a theoretical analysis of NVFP4 quantization and derive both lower and upper bounds for the required sweep range under mean square error (MSE) and weighted mean square error (WMSE) between the original tensor and the quantized reconstructed tensor. The proposed bounds substantially reduce the sweep space while preserving the optimal candidate, enabling negligible overhead compared with the baseline quantization operators.","pith_inferences":["Local per-block reconstruction error serves as a workable proxy for global model quality after quantization.","The bounding technique may be reused to shrink search costs when selecting scales in other fine-grained low-bit formats.","Coordinated optimization of scales across multiple tensor types (weights, activations, cache) can improve results when all are quantized at once."],"forward_implications":["ScaleSweep raises quantized accuracy above AbsMax initialization on Llama and Qwen models.","The method narrows the remaining gap to full-precision performance under end-to-end 4-bit quantization.","More than 93 percent of original model performance is retained when weights, activations, KV cache, and query states are quantized together.","The derived bounds keep added runtime cost negligible relative to standard quantization operators."],"fun_headline_variants":["ScaleSweep sweeps block scales within bounds to minimize NVFP4 error","Derived bounds cut sweep range for NVFP4 block scale optimization","Block scale sweeps under MSE WMSE improve NVFP4 LLM quantization","ScaleSweep reduces gap to full precision in NVFP4 quantized LLMs"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The block scale that minimizes per-tensor MSE or WMSE will also produce the highest accuracy once every component of the model has been quantized.","fun_headline_variants_meta":{"raw":{"variants":["ScaleSweep sweeps block scales within bounds to minimize NVFP4 error","Derived bounds cut sweep range for NVFP4 block scale optimization","Block scale sweeps under MSE WMSE improve NVFP4 LLM quantization","ScaleSweep reduces gap to full precision in NVFP4 quantized LLMs"]},"model":"grok-4.3","cost_usd":0.00617,"raw_usage":{"total_tokens":2914,"prompt_tokens":677,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":61699500,"prompt_tokens_details":{"text_tokens":677,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2164,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":677,"tokens_out":73,"duration_ms":17426,"temperature":1.0,"reasoning_tokens":2164,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T18:56:50.866463+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A measured case in which a scale lying outside the derived bounds yields lower end-to-end model error than every scale inside the bounds.","supporting_citations":[],"review_version":1}