{"id":"7fd4afa2-07a9-4c5e-9715-8f3775e91376","arxiv_id":"2606.18114","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"Ternary Mamba-2 1.3B models reach 48.1% zero-shot accuracy via QAT from pretrained checkpoints in 102M tokens, close to Bi-Mamba, with 3.61x compression.","lead":"This paper shows that ternary state space models can be created from existing pretrained checkpoints using grouped quantization-aware training and knowledge distillation, needing far less data than training from scratch. A smart generalist might read it to understand practical ways to shrink large AI models for use on phones or other small devices.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flags the abstract-only limitation and the post-hoc/collapse assumption as weakest; with no full text available for technical inspection, no additional load-bearing flaw can be isolated. The argument's internal logic holds on the provided summary.","tokens_in":1804,"tokens_out":251,"duration_ms":24768,"concrete_test":"Re-read the full manuscript (including § on zero-ratio collapse experiments and post-hoc ablation tables) and verify whether the reported 48.1% accuracy holds after explicit measurement of per-layer zero ratios and recurrence error propagation; if the mitigation details are absent or the post-hoc failure is shown only qualitatively, the claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract presents the central claim (pretrained QAT+KD suffices for ternary Mamba-2 at 102M tokens, post-hoc corrections fail due to SSM recurrence, and zero-ratio collapse is revealed but managed) as internally consistent with reported numbers and comparisons. No unsupported assumption, circularity, or missing derivation is detectable from the given text; the 1,000x token reduction and accuracy parity are stated with explicit baselines and CI.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that grouped quantization-aware training (QAT) with knowledge distillation from a frozen FP16 teacher, starting from a pretrained Mamba-2 1.3B checkpoint, enables effective W1.58A16 compression of state space models. This yields 3.61x size reduction (2687 MB to 744 MB) and 48.1% zero-shot accuracy on a 7-task average using only 102M tokens (4 GPU-hours on one H100), approaching Bi-Mamba's 48.4% within +/-0.9pp CI. The work identifies zero-ratio collapse as a novel instability arising from learnable quantization scales in the QAT-from-pretrained regime (absent in from-scratch training) and shows that post-hoc correction methods effective for Transformers fail for SSMs due to error accumulation through the recurrence. This reduces the marginal token budget by 1000x relative to prior from-scratch ternary SSM training on 150B tokens.","tokens_in":1867,"tokens_out":777,"duration_ms":34679,"significance":"If the empirical results and ablations hold, the work provides a practical, data-efficient path to ternary SSM deployment that avoids the prohibitive cost of from-scratch training. The 1000x token reduction, explicit comparison to Bi-Mamba with CI, and the identification of recurrence-specific quantization instabilities (zero-ratio collapse and failure of post-hoc fixes) are load-bearing contributions that could guide quantization research on recurrent architectures. The use of grouped QAT plus KD from pretrained checkpoints is a reproducible empirical finding with clear baselines.","major_comments":[{"comment":"§4.3 (Post-hoc correction experiments): the central claim that post-hoc strategies fail for SSMs due to recurrence error accumulation is load-bearing for arguing QAT-from-pretrained is necessary, yet the manuscript provides no quantitative measurement of state error propagation (e.g., via controlled injection of quantization noise into the recurrence and tracking of hidden-state drift over sequence length).","section":"§4.3"},{"comment":"Table 2 and §5.1 (zero-shot results): the 48.1% vs 48.4% comparison is reported with a +/-0.9pp CI, but the text does not specify the number of evaluation runs, task-level variance, or whether the CI accounts for multiple-testing across the 7 tasks; this weakens the claim of statistical parity.","section":"Table 2"},{"comment":"§3.2 (zero-ratio collapse definition): the novel instability is attributed to learnable quantization scales, but the manuscript does not include an ablation isolating the effect of learnable vs. fixed scales on the observed collapse, leaving open whether the phenomenon is specific to the grouped QAT formulation or more general.","section":"§3.2"}],"minor_comments":[{"comment":"The abstract and §2 reference 'Bi-Mamba' as a baseline without an explicit citation or description of its architecture/training details in the main text; this should be clarified for reproducibility.","section":null},{"comment":"Figure 3 (zero-ratio collapse visualization): the y-axis scaling and legend placement make it difficult to read the exact zero-ratio values at convergence; consider adding a table of final ratios.","section":"Figure 3"},{"comment":"The token budget comparison (150B vs 102M) assumes identical model size and task distribution between Slender-Mamba and the current experiments; a short note confirming this would strengthen the 1000x claim.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive recommendation of minor revision and the constructive comments. We address each major point below and commit to revisions that strengthen the manuscript without misrepresenting the existing results.","responses":[{"response":"We agree that a direct quantitative measurement of state error propagation would provide stronger support for the recurrence-specific claim. In the revised manuscript we will add a controlled experiment that injects synthetic quantization noise into the hidden states and tracks drift in hidden-state norms over increasing sequence lengths, with direct comparison to Transformer baselines.","revision_made":"yes","referee_comment":"[§4.3] §4.3 (Post-hoc correction experiments): the central claim that post-hoc strategies fail for SSMs due to recurrence error accumulation is load-bearing for arguing QAT-from-pretrained is necessary, yet the manuscript provides no quantitative measurement of state error propagation (e.g., via controlled injection of quantization noise into the recurrence and tracking of hidden-state drift over sequence length)."},{"response":"We will revise §5.1 and the caption of Table 2 to explicitly state the number of evaluation runs used to compute the CI, report task-level standard deviations, and confirm that the interval incorporates a correction for multiple testing across the seven tasks.","revision_made":"yes","referee_comment":"[Table 2] Table 2 and §5.1 (zero-shot results): the 48.1% vs 48.4% comparison is reported with a +/-0.9pp CI, but the text does not specify the number of evaluation runs, task-level variance, or whether the CI accounts for multiple-testing across the 7 tasks; this weakens the claim of statistical parity."},{"response":"We will add a targeted ablation in §3.2 that compares zero-ratio collapse when quantization scales are learnable versus held fixed (initialized from the pretrained checkpoint) under otherwise identical QAT-from-pretrained conditions. This will isolate the contribution of learnable scales.","revision_made":"yes","referee_comment":"[§3.2] §3.2 (zero-ratio collapse definition): the novel instability is attributed to learnable quantization scales, but the manuscript does not include an ablation isolating the effect of learnable vs. fixed scales on the observed collapse, leaving open whether the phenomenon is specific to the grouped QAT formulation or more general."}],"tokens_in":1609,"tokens_out":519,"duration_ms":36958,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that you can start from a pretrained Mamba-2 checkpoint, apply grouped quantization-aware training plus distillation, and reach 48.1% zero-shot accuracy on a 7-task average after only 102M tokens. That is roughly 1000x fewer tokens than the prior from-scratch ternary SSM work, and the model size drops from 2.7 GB to 744 MB. The paper also flags a zero-ratio collapse that appears when learnable scales are used in the SSM recurrence and shows that simple post-training fixes that work on Transformers do not transfer.\n\nWhat is new is the combination itself: taking an existing FP16 checkpoint and finishing the quantization in a data-efficient way rather than retraining everything from random weights. The observation that recurrence makes error accumulation different from attention-based models is useful to note even if it is not fully quantified yet.\n\nThe soft spots are the usual ones at this stage. The abstract gives the headline numbers and the 0.9 pp confidence interval, but there are no ablations on the grouping strategy, no learning curves, and no direct measurement of how much the recurrence actually amplifies quantization error compared with a Transformer baseline. The claim that post-hoc corrections fail rests on the reported outcome rather than a controlled comparison that isolates the recurrence effect. Those gaps are real but not fatal for a short paper; they are the kind of thing a referee would ask to see expanded.\n\nThis is for people already working on quantized SSMs or edge deployment who want a practical recipe rather than a theoretical advance. It is worth sending to review because the central empirical claim is stated clearly with a concrete baseline and the instability observation is at least falsifiable. A serious referee could tighten the experimental section without needing to rewrite the story.","headline":"Pretrained QAT gets ternary Mamba close to from-scratch results with 1000x less data, but the supporting experiments are still thin.","tokens_in":2362,"tokens_out":435,"would_cite":false,"duration_ms":12826,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A pretrained Mamba-2 checkpoint can be turned into a ternary model with grouped QAT and distillation using only 102 million tokens.","keywords":["state space models","mamba","quantization-aware training","ternary weights","knowledge distillation","model compression","recurrent models","edge deployment"],"falsifier":"Running the identical 102 million token budget from a random initialization instead of the pretrained checkpoint and finding that accuracy stays well below 48 percent.","tokens_in":2707,"feed_emoji":"📦","tokens_out":787,"duration_ms":27399,"temperature":0.7,"pith_summary":"The paper sets out to demonstrate that state space models do not require training from scratch to reach effective ternary quantization. Starting from an existing full-precision checkpoint and applying grouped quantization-aware training plus knowledge distillation from a frozen FP16 teacher produces a 3.61 times smaller model that retains nearly the same zero-shot performance. This matters for practical deployment because it slashes the data and compute needed by three orders of magnitude compared with prior from-scratch ternary SSM work. The same experiments uncover a training instability called zero-ratio collapse that is unique to the quantized SSM setting and show that correction methods borrowed from transformers do not transfer because of error buildup across the recurrence.","feed_headline":"Pretrained Mamba-2 reaches ternary form with 102M tokens","feed_subtitle":"Grouped QAT plus distillation yields 3.61x compression and 48.1% zero-shot accuracy, 1000x less data than from-scratch training.","key_machinery":"Grouped quantization-aware training that jointly optimizes quantization scales and model weights under a distillation loss from the frozen FP16 teacher, which counters zero-ratio collapse during the SSM recurrence.","core_discovery":"Grouped quantization-aware training with knowledge distillation from a frozen FP16 teacher applied to a pretrained Mamba-2 1.3B checkpoint yields a W1.58A16 model that occupies 744 MB instead of 2,687 MB and reaches 48.1 percent average zero-shot accuracy on seven tasks after 102 million tokens, coming within 0.9 percentage points of Bi-Mamba while using roughly one-thousandth the marginal token budget of from-scratch ternary training.","pith_inferences":["The same grouped QAT recipe may allow rapid adaptation of other pretrained SSM variants to ternary weights without repeating full pretraining.","Because the method works from an existing checkpoint, it opens the possibility of periodic on-device fine-tuning of compressed models when new data arrives.","The observed failure of transformer post-hoc fixes points to the need for recurrence-aware quantization analysis tools that are not yet standard.","If the 0.3 percentage point gap to Bi-Mamba can be closed with modest extra tokens, the approach would make ternary SSMs competitive for latency-critical applications."],"forward_implications":["Model size drops from 2,687 MB to 744 MB while zero-shot accuracy on the seven-task average reaches 48.1 percent.","Total training cost is limited to 4 GPU-hours on one H100.","Post-training correction methods that work for transformers produce large accuracy drops because errors accumulate through the SSM recurrence.","Zero-ratio collapse appears only in the QAT-from-pretrained regime and must be managed by the grouped scale updates.","The marginal token requirement falls by a factor of roughly 1,000 relative to the 150 billion tokens used in earlier from-scratch ternary SSM training."],"fun_headline_variants":["Mamba-2 to ternary in 102M tokens with grouped QAT","Pretrained Mamba-2 ternary in 102M tokens","Ternary Mamba-2 via QAT needs 102M tokens","QAT ternaryizes Mamba-2 pretrained with 102M tokens"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That the recurrent dynamics of the SSM allow the teacher signal to recover performance even after the quantization scales have been learned and the zero ratio has stabilized.","fun_headline_variants_meta":{"raw":{"variants":["Mamba-2 to ternary in 102M tokens with grouped QAT","Pretrained Mamba-2 ternary in 102M tokens","Ternary Mamba-2 via QAT needs 102M tokens","QAT ternaryizes Mamba-2 pretrained with 102M tokens"]},"model":"grok-4.3","cost_usd":0.006115,"raw_usage":{"total_tokens":2829,"prompt_tokens":712,"num_sources_used":0,"completion_tokens":77,"cost_in_usd_ticks":61153000,"prompt_tokens_details":{"text_tokens":712,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2040,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":712,"tokens_out":77,"duration_ms":15129,"temperature":1.0,"reasoning_tokens":2040,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T00:52:18.843017+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the identical 102 million token budget from a random initialization instead of the pretrained checkpoint and finding that accuracy stays well below 48 percent.","supporting_citations":[],"review_version":1}