{"id":"e18b0ce8-02ad-498a-ae9b-61d626bdffe3","arxiv_id":"2505.08620","paper_version":1,"verdict":"UNVERDICTED","confidence":"MODERATE","novelty_score":0.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey of post-training quantization techniques for large language models, covering schemes, granularities, and popular methods, with no new experimental results.","lead":"This paper is a review of post-training quantization, the practice of compressing large language models by lowering the numerical precision of their weights and activations. It organizes common methods, trade-offs, and supporting libraries for practitioners who want faster inference on limited hardware.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The HQQ description in Section 4.6 ('optimizes only for the zero-point and keeps the scaling factor static') is the weakest load-bearing claim; if inaccurate, the review's method coverage is misleading, and it is easy to check against HQQ's actual optimization loop.","rationale":"The reader's UNVERDICTED judgment is based on the paper being a review rather than a research contribution, and I agree that the review is not verifiable research. The strongest correctness risk within the review is the HQQ description, so I agree with the reader's choice of weakest assumption. If the test shows HQQ optimizes both parameters, the review misleads end-users about a method it explicitly recommends in the conclusion and Figure 10; that would strengthen the correctness-risk component, but it does not change the UNVERDICTED category, since the product remains a non-research review without new derivations or experiments. If the test shows scale is frozen, then no significant objection remains beyond the usual 'review not research' caveat. For these reasons I keep the verdict unchanged and propose a single inspection that settles the matter.","tokens_in":15221,"tokens_out":8452,"duration_ms":87948,"concrete_test":"Download the official HQQ repository (github.com/mobiusml/hqq), locate the half-quadratic optimization routine (e.g., 'optimize_weights' in hqq/core.py), and run it on a small random weight tensor with group size 128 and 4-bit quantization. Record the scale array before and after the optimization loop: if the scale tensor changes in any entry, Section 4.6's 'keeps the scaling factor static' is false. Also inspect the code inside the loop for an update rule that assigns scale, not only zero-point.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is to be a 'focused and high-level review' of PTQ methods, so the accuracy of its method summaries is load-bearing. The most exposed such summary is Section 4.6: 'HQQ optimizes, through a Half-Quadratic solver, only for the zero-point and keeps the scaling factor static.' The passage cites only the original HQQ paper/software, and it does not reproduce the HQQ update equations or quote the relevant section. Public descriptions of HQQ describe half-quadratic minimization of quantization error with respect to the quantization parameters; whether this includes the scale variable is exactly the point in dispute. If the actual HQQ algorithm updates the scale alongside the zero-point, then Section 4.6 misstates both what HQQ optimizes and why it is fast, and Figure 10's comparison loses context for practitioners. Conversely, if the HQQ code really does freeze the scale after initialization, this concern does not land. The claim is checkable, and it should not be accepted on the basis of the review alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a survey-style manuscript on post-training quantization (PTQ) for large language models. It introduces quantization fundamentals (symmetric and asymmetric schemes, static and dynamic quantization, parameter selection strategies, and quantization granularity) and then summarizes six commonly used PTQ methods: ZeroQuant, LLM.int8()/bitsandbytes, GPTQ, AWQ, SmoothQuant, and HQQ. It also provides a table of inference libraries and their supported quantization methods. The paper claims to be a focused, high-level, practitioner-oriented review rather than a source of new algorithms or experimental results.","tokens_in":15401,"tokens_out":5452,"duration_ms":56502,"significance":"If the method descriptions are accurate, this manuscript is a useful entry point for practitioners who want to understand the trade-offs among PTQ techniques without reading each method's original paper. Its strengths are the clean organization, the practical library-support table, the worked bits-per-weight overhead calculation, and the clear illustrations of quantization granularity. The paper makes no claim to new technical contributions, so its scientific value is entirely tied to the correctness and currency of its survey content; any substantive mischaracterization of a described method directly weakens the central claim of being a balanced and accurate review.","major_comments":[{"comment":"The sentence 'HQQ optimizes, through a Half-Quadratic solver, only for the zero-point and keeps the scaling factor static' is a load-bearing description of a major method and is not supported by the cited reference as given. Published descriptions of half-quadratic quantization minimize the quantization error with respect to the quantization parameters, including the scale, and the stated claim needs a direct citation to a specific equation or code path. Without that support, the passage misleads readers about what HQQ optimizes and why it is fast, which also affects the interpretation of Figure 10 and the concluding suggestion that HQQ represents a data-free approach. Please correct the sentence or add the exact optimization objective and update rules from the HQQ paper.","section":"4.6 (HQQ)"},{"comment":"The per-channel quantization example is built around attention heads ('Head 0 max abs val ... Head 2 max abs val'), but attention heads are not channels of a weight tensor in the sense used by per-channel quantization. Standard per-channel quantization assigns a scale (and optionally a zero-point) to each output channel of the linear layer, i.e., to individual rows of the weight matrix or dimensions of the output space. A single attention head spans a contiguous block of output dimensions and is therefore a group of channels, not a channel. The example should be reframed in terms of output-channel ranges, or it should be explicitly presented as group-wise quantization if head-level granularity is intended.","section":"3.8.2 (Quantizing per Channel)"},{"comment":"The description of dynamic quantization conflates it with calibration. The text says dynamic quantization 'attempts to solve this by running inference for a few different inputs to observe the typical values (min, max)' and 'should perform higher than static.' Dynamic quantization computes the min/max range of each activation tensor at runtime for the actual input, not from a small set of pre-run calibration inputs. It is generally more accurate than static quantization because it adapts to the current input, but it incurs per-tensor runtime overhead. Please rewrite this paragraph to distinguish runtime range computation from calibration.","section":"3.6 (Dynamic Quantization)"}],"minor_comments":[{"comment":"In Eq. (2), the zero-point formula z = -1 * beta / s omits the rounding operation. As written, z can be fractional, while the quantized values in Eq. (3) are integers. Add an explicit rounding or nearest-integer operation, or state that z is rounded before use.","section":"3.3 (Asymmetric Quantization)"},{"comment":"The sentence beginning 'There is currently no agreed upon best practice for PTQ strategies, and performance across different models (like Gemma..., Llama...' is incomplete; it lacks a predicate and ends with a reference. It should be completed or broken into two separate sentences.","section":"5 (Conclusion)"},{"comment":"There are several typographical and spacing errors that should be corrected in a final revision: 'matrix multiplcations' in the Introduction, 'commong' in Section 3.2, 'with with hardware-optimized' in Section 3.2, the missing space in 'otheremergent' in the abstract/Introduction, and 'as reported byHQQ' in the Figure 10 caption.","section":"Throughout"},{"comment":"The method name 'AWQ' is typeset as 'A WQ' in the Section 4.4 heading and in several rows of Table 1. This is distracting and should be made consistent with the standard spelling 'AWQ'.","section":"4.4 and Table 1"}],"recommendation":"major_revision","confidential_remarks":"This is a survey paper whose value depends entirely on the accuracy of its summaries. The three major comments are all fixable without changing the paper's scope, and the rest of the manuscript is a reasonable high-level overview. I would not reject on novelty grounds, but the HQQ description in Section 4.6 must be corrected or substantiated with the actual optimization equations before the paper can be accepted as a reliable reference. The paper should also state its coverage cutoff date more explicitly, since the quantization landscape is moving quickly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a review paper, not a research contribution. It explicitly positions itself as a 'high-level review' of post-training quantization for LLMs, and there is no new method, experiment, or derivation in it. If you treat it as a survey, it is mostly clear and well-organized for someone new to the area: the background on symmetric/asymmetric quantization, granularities, and outlier handling is a decent orientation, and the library table is useful. I'd give it credit for bringing together the major PTQ methods (GPTQ, AWQ, SmoothQuant, ZeroQuant, bitsandbytes, HQQ) in one place with availability notes.\n\nThe soft spots are real and some are load-bearing. The most exposed is Section 4.6's claim that HQQ 'optimizes, through a Half-Quadratic solver, only for the zero-point and keeps the scaling factor static.' That is almost certainly wrong: HQQ uses half-quadratic splitting to optimize the quantization parameters—both scale and zero-point—iteratively. This misstates what HQQ does and why it is fast, and Figure 10's comparison loses context. It is checkable from the cited HQQ paper and code, and the review doesn't reproduce the update equations, so it should not be accepted at face value.\n\nThere are also conceptual slips in the background. Section 3.8.2 illustrates per-channel quantization using attention heads as if a head were a channel; in transformer weight matrices, channels are rows, and heads span multiple rows. The example is misleading. Section 3.6 describes dynamic quantization as 'running inference for a few different inputs to observe typical values,' which sounds like calibration rather than per-input runtime range computation. And the conclusion literally trails off mid-sentence when comparing models, which suggests the manuscript wasn't polished.\n\nOn the positive side, the paper doesn't overclaim: it stays within the review scope, cites the relevant literature widely, and generally describes the methods' trade-offs in a fair way. The GPTQ and AWQ summaries are accurate enough. So it's not a dishonest paper; it's an uneven one with a few factual mistakes.\n\nWho is this for? A beginner or practitioner who wants a quick orientation and has the patience to double-check the details. A researcher in the field won't learn much new.\n\nMy recommendation: desk reject, but with an invitation to resubmit after a careful fact-check pass, especially on HQQ, the per-channel example, and the dynamic quantization description. As it stands, I wouldn't send it to a serious referee; the errors undermine its value as a reference.","headline":"A readable but flawed survey of PTQ for LLMs; the HQQ summary is likely wrong and the review needs a fact-check before it can be trusted as a reference.","tokens_in":15880,"tokens_out":4416,"would_cite":false,"duration_ms":41791,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This review argues that post-training quantization lets end-users shrink large language models to a few bits per weight, making them faster and more accessible without retraining.","keywords":["post-training quantization","large language models","inference efficiency","integer quantization","activation outliers","weight-only quantization","model compression"],"falsifier":"Run the HQQ quantization method on a small open-weight model while recording the learned scale parameter before and after optimization; if the scale changes along with the zero-point, the paper's Section 4.6 description is contradicted.","tokens_in":2508,"feed_emoji":"📉","tokens_out":5582,"duration_ms":118714,"temperature":0.7,"pith_summary":"This paper is a focused, high-level review of post-training quantization (PTQ) for large language models, with the explicit aim of helping end-users make models faster and more accessible on ordinary hardware. It argues that a practical understanding of the quantization pipeline—linear uniform mapping, symmetric versus asymmetric ranges, static versus dynamic calibration, and per-tensor, per-channel, or per-group granularity—is enough to deploy weight-only 4-bit and other PTQ schemes found in popular inference libraries. The review also highlights that activation outliers are the central difficulty and that quality degrades quickly for extreme low-bit methods below about 3 bits per weight. If the review is accurate, a practitioner can meaningfully choose a quantization configuration for a given model and hardware without needing to retrain.","feed_headline":"Post-training quantization shrinks LLMs without retraining","feed_subtitle":"A focused review shows how choosing bit width, granularity, and calibration trades off size, speed, and accuracy.","key_machinery":"The central object is linear uniform quantization, defined by the mapping $x_q = \\operatorname{clamp}(\\lfloor x_f / s \\rfloor + z,\\, 0,\\, 2^n-1)$ with dequantization $x_f \\approx s \\cdot (x_q - z)$. The scaling factor $s$ sets the range, the zero-point $z$ shifts it, and $n$ is the bit width; symmetric quantization drops the zero-point. This one identity carries the whole review: every method it surveys is a different way of choosing $s$, $z$, and the granularity at which they are shared, and every trade-off involving outliers, overhead, latency, and effective bits per weight is expressed in those terms.","core_discovery":"The paper's central claim is that post-training quantization is the practical, end-user-focused route to faster and more accessible LLM inference, and that this route is best understood through the quantization parameter pair $(s,z)$ and the choices around it. It presents PTQ as a mapping of weights and activations to low-bit integers, with a scaling factor and a zero-point defining the range, and organizes the field by scheme, calibration, parameter-selection strategy, granularity, and the major methods including 8-bit outlier decomposition, GPTQ, AWQ, SmoothQuant, and HQQ. The review claims that 4-bit weight-only quantization is a popular and well-supported practical setting, that extreme low-bit methods below 3 bits degrade quickly, and that no single best practice currently exists. It is not proposing a new algorithm; it is arguing that this organizing frame lets an end-user choose a quantization configuration for a given model and hardware target.","pith_inferences":["A practical takeaway the paper leaves implicit: the method table is a decision aid, so an end-user with a single GPU can narrow the options to one 8-bit scheme and two or three 4-bit weight-only methods, then test on the actual task.","A testable extension would be to compare the review's 4-bit default against 3-bit and 5-bit settings on instruction-following or reasoning benchmarks rather than perplexity alone, since the review itself notes that perplexity may not reflect task quality.","The review's emphasis on data-free HQQ suggests that calibration-free quantization could become the default for end-user use, but only if the method's actual optimization behavior matches the description given."],"forward_implications":["End-users can run models once stored in hundreds of gigabytes on a single consumer GPU by choosing weight-only 4-bit quantization, a setting the review identifies as popular and well-supported.","Going below about 3 bits per weight is not currently a dependable option: the review reports that quality degrades quickly in that regime.","Granularity is the main control knob: per-tensor quantization minimizes overhead, per-channel and per-group quantization preserve accuracy, and the overhead appears as extra bits per weight.","Handling activation outliers is the common theme connecting the main PTQ methods; each method's differentiating move is how it treats the outlier channels.","No single best practice exists, so evaluating the chosen quantization on the target model and task is part of using any of these methods."],"supporting_citations":[{"why":"Supplies the 8-bit LLM.int8() method and the observation that activation outliers appear in every query, key, and value projection.","marker":"Dettmers et al., 2022"},{"why":"Describes ZeroQuant and its W8A8 and mixed-precision PTQ approach, forming the basis of the first method section.","marker":"Yao et al., 2023"},{"why":"Introduces GPTQ, the one-shot weight-only 3- and 4-bit method the review presents as a standard end-user option.","marker":"Frantar et al., 2023"},{"why":"Introduces AWQ, the activation-aware per-channel weight quantization method described in the review.","marker":"Lin et al., 2024a"},{"why":"Introduces SmoothQuant, the W8A8 method that smooths activation outliers through channel-wise scaling.","marker":"Xiao et al., 2024"},{"why":"Introduces HQQ, the data-free Half-Quadratic quantization method whose description carries the review's most fragile assumption.","marker":"Badri and Shaji, 2023"},{"why":"Supports the trade-off between finer quantization granularity and added overhead in per-group settings.","marker":"Heo et al., 2024"},{"why":"A low-bit quantization survey that bounds the review's scope and points readers to extreme low-bit methods.","marker":"Gong et al., 2024"}],"fun_headline_variants":["Quantize your LLM: size down, speed up, no retraining","LLM quantization: the field, the flavors, and trade-offs","Post-training quantization: the end-user's toolkit","4-bit weight-only: a practical sweet spot for LLMs","Shrink LLMs fast: the quantization trade-off guide"],"cache_read_input_tokens":18176,"weakest_assumption_plain":"The load-bearing premise is that the paper's descriptions of the surveyed PTQ methods are accurate; the fragile case is its claim that HQQ optimizes only the zero-point while keeping the scale static, which may contradict how the method actually works.","fun_headline_variants_meta":{"raw":{"variants":["Quantize your LLM: size down, speed up, no retraining","LLM quantization: the field, the flavors, and trade-offs","Post-training quantization: the end-user's toolkit","4-bit weight-only: a practical sweet spot for LLMs","Shrink LLMs fast: the quantization trade-off guide"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00129,"raw_usage":{"total_tokens":5190,"prompt_tokens":790,"completion_tokens":4400,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":406,"completion_tokens_details":{"reasoning_tokens":4314}},"tokens_in":406,"tokens_out":4400,"duration_ms":35869,"temperature":1.0,"reasoning_tokens":4314,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:49:15.280190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the HQQ quantization method on a small open-weight model while recording the learned scale parameter before and after optimization; if the scale changes along with the zero-point, the paper's Section 4.6 description is contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 8-bit LLM.int8() method and the observation that activation outliers appear in every query, key, and value projection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the trade-off between finer quantization granularity and added overhead in per-group settings."}],"review_version":1}