{"id":"65f2ec39-c571-4bf8-b4b8-5274ba795c2f","arxiv_id":"2505.09188","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A structured survey that categorizes zero-shot quantization methods into synthesis-free, generator-based, and noise-optimization approaches, with a side-by-side accuracy comparison.","lead":"This paper surveys methods for compressing trained neural networks without access to their original training data, a problem called zero-shot quantization. It sorts the methods into three families and compares their reported accuracy, which helps engineers pick a data-free compression approach.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 mixes incompatible evaluation settings (ResNet-18 W4A4 vs DeiT-Tiny W8A8, differing image counts), and the DeepInversion row appears mislabeled; the survey's comparative accuracy analysis is not reliable as presented.","rationale":"The reader's weakest assumption points directly to the comparability of Table 1, and my read confirms this as the most load-bearing concern for the paper's central claim. The survey aims to be the first in-depth ZSQ survey and provides a categorization plus a performance comparison. The categorization may be informative, but the comparative table is a key deliverable. Mixing ResNet-18 and DeiT-Tiny results, different bit-widths, and different image-count conventions makes the table internally inconsistent and misleading. The specific DeepInversion mislabeling is an additional concrete error that strengthens the concern. This is not a case of disagreement with the field consensus; it is an internal inconsistency in the manuscript's own reporting. Because the reader already assigned a CONDITIONAL verdict based on this issue, my concern does not move the verdict; it reinforces the need for correction before the survey can be trusted as a reference. No ad hominem is intended; the issue is in the table's construction and labeling, not in the authors' integrity. A focused re-verification of Table 1 would settle whether the concern fully lands, and if most entries survive, the survey could be accepted after fixing the footnotes and separating the comparisons by setting.","tokens_in":13316,"tokens_out":4516,"duration_ms":39195,"concrete_test":"Re-verify every row of Table 1 against the cited original papers, recording for each method: base model, bit-width (weights/activations), image count, and the reported accuracy. Specifically, look up DeepInversion's original ResNet-18 W4A4 and W3A3 accuracy; if the numbers 70.27/64.28 actually come from DeiT-Tiny W8A8/W4A8, correct the row. Then group rows by (model, bit-width, #images) and recompute within-group rankings. If any method's relative standing changes, or if any entry cannot be verified from the source, the table cannot serve as a dependable performance baseline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The survey's central contribution includes a comparative performance table (Table 1) that is meant to benchmark ZSQ methods. However, the table conflates heterogeneous evaluation settings. The caption claims accuracy on ResNet-18 W4A4/W3A3, but rows for PSAQ-ViT, SMI, and CLAMP-ViT report DeiT-Tiny W8A8/W4A8 accuracy via footnotes. The '# Images' column mixes totals (1.28M for generator-based QAT), per-class or dataset sizes (32 for ViT PTQ, 1K for CNN PTQ), and calibration set sizes (5.12K for noise-optimization QAT). More concerning, the DeepInversion row carries footnotes (∗/†) defined solely for DeiT-Tiny, even though DeepInversion is a CNN method and its original ResNet-18 results are not W8A8/W4A8 Accuracy of DeiT-Tiny; this row appears to be mislabeled. Similarly, GenQ's 1K§ with footnote '§ PTQ setting' does not match the section 5.2 description of diffusion-based generation. Because the table is the basis for the survey's 'compare their performance' and 'performance baseline' claims, these inconsistencies invalidate any cross-method ranking drawn from it. The taxonomy and qualitative descriptions may remain useful, but the quantitative analysis as presented cannot support the claim of a comprehensive, reliable comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript surveys zero-shot quantization (ZSQ), a setting in which a pre-trained network is quantized without access to real training data. It formalizes the ZSQ problem, identifies three main challenges, and organizes existing methods into a three-way taxonomy: synthesis-free, generator-based, and noise-optimization-based approaches. For each method, the paper summarizes the motivation, core idea, and reported findings, and it provides a comparison table intended to serve as a performance baseline. The descriptive and taxonomic content is largely coherent and appears to align with the cited papers, but the quantitative comparison in Table 1 mixes incompatible evaluation settings and contains labeling inconsistencies. As presented, the table cannot support the paper's comparative-performance claims, which weakens one of the stated contributions.","tokens_in":13764,"tokens_out":4739,"duration_ms":46962,"significance":"If the descriptive content is taken on its own terms, this survey is a useful entry point for researchers entering ZSQ: it provides a formal problem definition, a clear and sensible three-way categorization with a helpful schematic (Figure 1), detailed summaries of representative methods, and a reasonable set of future directions. The paper also attempts a quantitative baseline table, and that part currently fails because the rows are not commensurable: ResNet-18 W4A4 numbers are mixed with DeiT-Tiny W8A8/W4A8 numbers, and the '# Images' column conflates totals, per-class counts, and calibration-set sizes. The 'first in-depth survey' claim is plausible but would be stronger with an explicit statement of search scope and inclusion criteria. Self-citations such as SynQ appear in the table, but the taxonomic and descriptive claims do not depend on those self-cited results, so circularity is not a serious concern.","major_comments":[{"comment":"The cross-method comparison in Table 1 is not valid as presented. The caption states that the table reports W4A4/W3A3 accuracy for ResNet-18 on ImageNet, yet the rows for PSAQ-ViT, SMI, and CLAMP-ViT report W8A8/W4A8 accuracy for DeiT-Tiny via footnotes. The '# Images' column also mixes total generator-training sample counts (1.28M), total synthesized-sample counts (5.12K), calibration-set sizes (1K), and ViT-specific set sizes (32). Because these quantities are not commensurable, any ranking or 'compare their performance' conclusion drawn from Table 1, as advertised in the Introduction and in Section 3, is not supported. I recommend restructuring the table into per-setting blocks, clearly stating the source of every number (original paper versus re-implementation), or removing the quantitative comparison and presenting the table as a qualitative feature summary only.","section":"Table 1 and Section 3"},{"comment":"The DeepInversion row carries the footnotes * and dagger, which are defined only for W8A8/W4A8 accuracy of DeiT-Tiny. DeepInversion is described in Section 6.1 as a CNN-oriented method with ResNet-18 experiments, so attaching DeiT-Tiny footnotes to it is misleading and makes the reported 70.27/64.28 numbers ambiguous. The row should either be relabeled with the actual evaluation protocol or removed from the footnoted group. The same ambiguity affects the GenQ and MixMix rows, where the annotation '1K (section sign)' is explained only as 'PTQ setting'; please clarify whether 1K is the calibration-set size or the size of the synthetic dataset, especially for GenQ, which Section 5.2 describes as using diffusion-based synthesis.","section":"Table 1, DeepInversion row"},{"comment":"The manuscript states that it evaluates ResNet-18 accuracy 'for fair benchmarking across methods' but does not say whether the numbers were taken from the original papers or produced by a unified re-implementation, nor which quantization configuration was used (per-channel versus per-tensor, symmetric versus asymmetric, calibration protocol, etc.). Without this information, even internally consistent rows cannot be independently checked, and the fairness claim in Section 3 is not verifiable. Please add a methodology paragraph or detailed table footnote describing the exact protocol, including how the ViT rows were obtained and why a ResNet-18 header applies to them.","section":"Section 3, evaluation methodology"}],"minor_comments":[{"comment":"The heading 'QA T and PTQ' contains an unintended space and should read 'QAT and PTQ'.","section":"Section 2.1, heading"},{"comment":"The statement that PTQ methods are 'typically evaluated with sets of 1,000 for CNNs and 32 for ViTs' is difficult to reconcile with Table 1, where noise-optimization-based QAT rows show 5.12K images; please clarify which numbers are calibration-set sizes and which are totals.","section":"Section 6.2, first sentence"},{"comment":"In-text citation formats are inconsistent, for example 'GenQ[2024b]' and 'AdaSG[2023b]' versus '[Li et al., 2024b]'; please standardize to a single style.","section":"References and in-text citations"},{"comment":"The estimate '1 to 4 RTX 4090 GPU hours' for generating 5,120 images should specify the method and exact setting to which it refers.","section":"Section 7, 'Faster generation of synthetic datasets'"},{"comment":"The notation 'WBAB' in the caption is used without definition; please spell out that W4A4 means weights and activations quantized to 4 bits, and clarify whether the notation applies only to weight-activation quantization.","section":"Table 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computer-vision or efficient-deep-learning venue, and the descriptive survey content is solid enough to be worth revising. The main risk is the quantitative comparison table: as it stands, it could mislead readers and draws attention away from the useful taxonomy. I also suggest the authors verify that they are not missing a previously published 'data-free quantization' survey, since the 'first in-depth survey' claim is a prominent part of the abstract. The self-citation level is noticeable but not disqualifying; however, including the authors' own SynQ in the benchmark table without a clear protocol invites extra scrutiny from reviewers and readers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this survey gives the ZSQ field a workable taxonomy and a readable map of methods, and that's genuinely useful. The three-way split (synthesis-free, generator-based, noise-optimization-based) plus QAT/PTQ is the right organizing principle, and the per-method descriptions track the original papers well. The future-work section is thoughtful. The claim to be the first in-depth ZSQ survey looks plausible.\n\nThe soft spot is Table 1. The table is meant to let readers compare methods side-by-side, but it mixes ResNet-18 W4A4 numbers with DeiT-Tiny W8A8/W4A8 numbers under one accuracy header, with only footnotes to separate them. The DeepInversion row carries footnotes that only make sense for DeiT-Tiny, which can't be right for a CNN method. The '# Images' column is a grab-bag: 1.28M for generator QAT, 5.12K for noise QAT, 1K for PTQ, 32 for ViT, and per-class vs total isn't distinguished. GenQ's 1K with 'PTQ' footnote doesn't square with the diffusion-based description in Section 5.2. As a result, any ranking you'd draw from the table is not reliable. This is a genuine flaw, not a quibble, because the survey's 'compare their performance' claim rests on that table.\n\nThe taxonomy and qualitative content are solid enough that the paper deserves a serious referee. I'd send it out, but with a clear instruction: fix Table 1 so CNNs and ViTs are separated, bit-widths are explicit per method, and the image-count column has a consistent definition. Also verify the DeepInversion entry. The self-citations are minor and the survey's argument doesn't depend on them.\n\nWho gets value: anyone entering zero-shot quantization who wants a structured reading list, and researchers looking for open problems. After the table is cleaned up, I'd trust it as a reference. It's not a technical breakthrough, but it's a legitimate survey contribution.","headline":"A useful first ZSQ survey with a sound taxonomy, but its comparison table mixes incompatible settings and needs correction before the quantitative claims can be trusted.","tokens_in":14101,"tokens_out":1895,"would_cite":true,"duration_ms":17482,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Without any real data, zero-shot quantization compresses neural networks; this survey sorts the field into three families.","keywords":["zero-shot quantization","data-free quantization","model compression","post-training quantization","synthetic data generation","network quantization","vision transformers","survey"],"falsifier":"Re-run three representative methods, one from each family, under a single fixed protocol: the same ResNet-18 model, the same W4A4 bit-width, the same number of synthetic images, and the same evaluation split. If the accuracy ordering differs from the ordering implied by Table 1, the survey's comparative conclusions depend on the heterogeneous settings rather than on the methods themselves.","tokens_in":13148,"feed_emoji":"⚙️","tokens_out":5985,"duration_ms":51243,"temperature":0.7,"pith_summary":"Zero-shot quantization (ZSQ) is the attempt to compress a trained neural network into low-bit arithmetic without using any of the data it was trained on, which matters when data is private, proprietary, or regulated. The paper claims to be the first in-depth survey of this area, and it supports that claim with a formal problem definition, three named challenges, and a categorization of methods by how they obtain synthetic data: synthesis-free, generator-based, and noise-optimization-based. A reader gets a map of the field, a comparison table of reported accuracies, and a list of open problems, including extending ZSQ beyond image classification and toward 3-bit or lower precision.","feed_headline":"Survey organizes zero-shot quantization into three families","feed_subtitle":"A field map and comparison table show how models are quantized when training data is off-limits.","key_machinery":"The load-bearing object is the three-way taxonomy of data acquisition in zero-shot quantization: synthesis-free methods, which recover accuracy without any generated data; generator-based methods, which train a GAN-style generator to produce synthetic examples; and noise-optimization-based methods, which directly optimize input noise through a two-step pattern of dataset synthesis followed by model quantization. Each family splits further into zero-shot quantization-aware training (QAT) and zero-shot post-training quantization (PTQ). The taxonomy does the work of making heterogeneous methods comparable and exposing shared baseline losses, such as batch-normalization statistics loss for CNNs and patch similarity entropy loss for vision transformers.","core_discovery":"The organizing discovery is that the entire ZSQ literature can be classified by data-generation strategy rather than by quantization technique. Synthesis-free methods never create data; they repair quantization damage using structural properties such as cross-layer weight equalization, Hessian approximations, or closed-form adjustment after pruning. Generator-based methods train a separate generative model to produce synthetic images for fine-tuning or calibration. Noise-optimization-based methods skip the generator and directly optimize random noise, usually by matching batch-normalization statistics and class predictions of the pre-trained model, with post-training variants needing as few as 32 images for vision transformers. The paper presents this taxonomy as its principal contribution, alongside a unified problem statement and a performance comparison table.","pith_inferences":["If the taxonomy is right, the practical bottleneck of ZSQ is synthetic-data quality rather than quantization itself, so advances in diffusion-based image generation should transfer directly to ZSQ performance.","The comparison table mixes CNN and ViT results, different bit-widths, and different synthetic-data budgets, so cross-method rankings are suggestive only; a standardized benchmark with fixed model, bit-width, and sample budget would settle which family is best.","The paper's image-classification focus suggests a ready test of the taxonomy's generality: zero-shot quantization of language models, which lack batch-normalization statistics, may force a fourth family or a new baseline loss."],"forward_implications":["A newcomer can identify which ZSQ family a method belongs to and which baseline losses it builds on, without reading every primary paper.","The comparison table shows noise-optimization-based PTQ and synthesis-free methods reaching accuracy close to generator-based QAT while using far fewer synthetic images.","Because BNS loss anchors CNN methods while ViT methods need attention-based losses like patch similarity entropy, architecture-specific ZSQ development will continue.","The paper's named future directions include theoretical bounds on data-free accuracy, faster synthetic-data generation, and combining quantization with pruning or low-rank compression."],"supporting_citations":[{"why":"DFQ is the first synthesis-free ZSQ method and supplies the cross-layer equalization and bias-correction baseline.","marker":"[Nagel et al., 2019]"},{"why":"DeepInversion founds the noise-optimization family with BNS-based dataset synthesis and knowledge transfer.","marker":"[Yin et al., 2020]"},{"why":"ZeroQ establishes zero-shot PTQ and the small-calibration-set regime.","marker":"[Cai et al., 2020]"},{"why":"GDFQ is the first generator-based ZSQ method and sets the alternating optimization recipe.","marker":"[Xu et al., 2020]"},{"why":"PSAQ-ViT is the first ViT-specific ZSQ method and introduces patch similarity entropy as the ViT baseline loss.","marker":"[Li et al., 2022]"},{"why":"KW introduces class-specific synthetic generation with the BNS-plus-inversion loss combination used widely.","marker":"[Haroush et al., 2020]"},{"why":"This existing quantization survey supplies the broader context that the paper's ZSQ-specific taxonomy extends.","marker":"[Gholami et al., 2022]"}],"fun_headline_variants":["Zero-shot quantization: a survey maps three data-free paths","How to quantize without data: three strategies surveyed","Survey classifies zero-shot quantization by data generation","Data-free quantization: taxonomy of three method families","Zero-shot quantization: first survey organizes three families"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison table's ability to rank methods assumes that accuracy numbers taken from different papers are directly comparable, even though the papers use different models, bit-widths, and synthetic-image budgets, so relative standing could change under a uniform protocol.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot quantization: a survey maps three data-free paths","How to quantize without data: three strategies surveyed","Survey classifies zero-shot quantization by data generation","Data-free quantization: taxonomy of three method families","Zero-shot quantization: first survey organizes three families"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1183,"prompt_tokens":826,"completion_tokens":357,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":283}},"tokens_in":442,"tokens_out":357,"duration_ms":3634,"temperature":1.0,"reasoning_tokens":283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:36:49.883899+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run three representative methods, one from each family, under a single fixed protocol: the same ResNet-18 model, the same W4A4 bit-width, the same number of synthetic images, and the same evaluation split. If the accuracy ordering differs from the ordering implied by Table 1, the survey's comparative conclusions depend on the heterogeneous settings rather than on the methods themselves.","supporting_citations":[{"cited_title":"Data-free quantization through weight equalization and bias correction","cited_arxiv_id":null,"evidence_quote":"DFQ is the first synthesis-free ZSQ method and supplies the cross-layer equalization and bias-correction baseline."},{"cited_title":"Dreaming to distill: Data-free knowl- edge transfer via deepinversion","cited_arxiv_id":null,"evidence_quote":"DeepInversion founds the noise-optimization family with BNS-based dataset synthesis and knowledge transfer."},{"cited_title":"Ze- roq: A novel zero shot quantization framework","cited_arxiv_id":null,"evidence_quote":"ZeroQ establishes zero-shot PTQ and the small-calibration-set regime."},{"cited_title":"The knowledge within: Meth- ods for data-free model compression","cited_arxiv_id":null,"evidence_quote":"KW introduces class-specific synthetic generation with the BNS-plus-inversion loss combination used widely."},{"cited_title":"A survey of quantization methods for efficient neural network inference","cited_arxiv_id":null,"evidence_quote":"This existing quantization survey supplies the broader context that the paper's ZSQ-specific taxonomy extends."}],"review_version":1}