{"id":"adfd6123-e7da-44a3-95b4-1a657a89301c","arxiv_id":"2608.09595","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Interleaving cross-block quantization so seam pairs are refined twice reduces perplexity and prevents divergence in low-bit LLM compression, with a conditional bound showing seams multiply an error contraction factor.","lead":"This paper introduces a scheduling tweak for compressing large language models to around 2 bits or less, where boundary pairs between chunks of Transformer blocks get refined twice instead of once. The change consistently lowers perplexity in experiments and rescues cases where the previous sweep order collapses, which matters for cheaper deployment of large models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical seam attribution is confounded: ICBQ adds re-rolls and extra pair-updates versus Sequential CBQ, so the observed gains may not come from seam revisits.","rationale":"The reader correctly identifies Assumption B.2's uniform per-sample contraction as an unverified theoretical premise; the paper's own Appendix B.11 admits only an aggregate gamma diagnostic, and Remark B.3 concedes that rollback does not certify Eq. (21). That is a genuine limitation of the theory. However, the central claim of the paper is the empirical improvement of ICBQ over Sequential CBQ, and that observation does not depend on Assumption B.2 being proven; the theorem is explicitly a bound-to-bound comparison under stated assumptions. The more load-bearing threat to the central claim is that the empirical improvement may be caused by confounds that are not isolated in any experiment: ICBQ adds re-rolls and extra pair-updates relative to the baseline. The K-ablation changes seams, re-roll count, and compute together, so it cannot identify the seam as the active ingredient. This is a concrete, testable gap in causal attribution, and it is not mentioned in the paper's limitations section. Because the reader's verdict is already CONDITIONAL, my concern does not move the verdict; it sharpens the condition: the authors should add controls that separate seam revisits from re-rolls and from total refinement budget, and should also seek a uniform per-sample contraction certificate if the theoretical mechanism is to be claimed. I therefore recommend keeping the CONDITIONAL verdict, with the additional condition made explicit.","tokens_in":38153,"tokens_out":9379,"duration_ms":85230,"concrete_test":"Run a Sequential CBQ control that mirrors ICBQ's re-roll schedule: after every K blocks, re-roll student/teacher streams from saved layer-0 inputs and continue the left-to-right pair sweep, but process each pair exactly once (no second seam pass). Compare C4 perplexity against Sequential CBQ and ICBQ on the seven-model set. Additionally, run Sequential CBQ with two complete left-to-right sweeps (or a sweep plus reverse sweep) so the total pair-update count equals ICBQ's L-1+S. If the re-roll-only or compute-matched baseline matches ICBQ within noise, the observed gains are not attributable to seam revisits, and the theoretical gamma^S mechanism is not supported as the active explanation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is that revisiting seam pairs (the interleaved schedule) causes the perplexity improvement over Sequential CBQ. But ICBQ differs from the Sequential CBQ baseline in two ways beyond the seam revisit: (i) activations are re-rolled from saved layer-0 inputs at every chunk boundary (Algorithm 1, step 12; Algorithm 2, step 1), while Sequential CBQ performs no re-roll; and (ii) ICBQ performs more total CBQ pair-updates: Sequential does L-1 = 31 pair refinements for L=32, while ICBQ with K=4 does L-1+S = 38, with measured wall-clock ratio ~1.21x (Table 5). The ablation in Table 3 varies K, which simultaneously changes the number of seams, the number of re-rolls, and the total refinement budget. Thus the monotonic improvement with smaller K is equally consistent with 'more re-rolls' or 'more compute' being the active ingredient. Section 5.4's claim that 'The seam is the story' is not established, because no experiment holds re-roll frequency or total pair-update count fixed while varying only seam revisits. If a Sequential CBQ variant with the same periodic re-rolls but no second seam pass, or with a matched total budget of repeated sweeps, reproduces ICBQ's gains, then the proposed seam mechanism, and the gamma^S amortization story, are not the cause of the observed improvement. This is load-bearing because the paper's contribution is specifically the interleaved seam schedule, not merely extra refinement passes or periodic activation refreshment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Interleaved Cross-Block Quantization (ICBQ), a scheduling change to block-wise post-training quantization in which each chunk-boundary pair ('seam') is refined twice: once at the end of one chunk and again at the start of the next. The method keeps the local two-block CBQ objective and reuses calibration inputs, adding only a constant-size layer-0 activation buffer. Under stated Lipschitz, per-sample contraction, and exact re-roll assumptions, the authors derive an upper-bound comparison (Theorem 4.1 / Theorem B.5) in which each seam multiplies the propagated error term by an additional gamma factor, leading to an amortization gain prediction gamma^{-S}. Experiments on seven base models with ternary DBF and GPTQ W3/W2g128 report lower perplexity than a matched Sequential CBQ baseline, including cases where the baseline diverges, and an ablation varying chunk size K shows monotonic improvements as K decreases. The paper concludes that schedule design, specifically seam revisits, is the active ingredient.","tokens_in":38452,"tokens_out":5219,"duration_ms":49085,"significance":"If the empirical claims hold, ICBQ is a genuinely useful and cheap scheduling modification for extreme low-bit weight-only PTQ: it introduces no new objective, no architectural change, and only a constant-size memory buffer, while improving perplexity and zero-shot accuracy over a matched Sequential CBQ baseline across seven models and two inner quantizers. The paper is also commendable for shipping detailed algorithms, full proofs in the appendix, a numerical toy simulation, and a clear statement of the conditional nature of the theoretical result. However, the significance is bounded by two gaps: the theoretical contraction mechanism rests on Assumption B.2, which the authors explicitly say is not certified empirically in the per-sample uniform form required; and the empirical 'seam is the story' claim is confounded by concurrent changes in activation re-rolls and total refinement computation. Both points are addressable, but they are central to the paper's contribution as stated.","major_comments":[{"comment":"The empirical isolation of the seam effect is confounded. ICBQ differs from the Sequential CBQ baseline in at least three ways: (i) it performs periodic activation re-rolls from saved layer-0 inputs, which Sequential CBQ does not do; (ii) it performs more total pair-refinement updates (L-1+S vs. L-1, with measured wall-clock ratio ~1.21x in Table 5); and (iii) it revisits seam pairs. The chunk-size ablation in Table 3 varies K, which simultaneously changes the number of seams, the number of re-rolls, and the total refinement budget. The monotonic improvement with smaller K is therefore equally consistent with 'more re-rolls' or 'more compute' being the active ingredient rather than the seam revisit itself. Section 5.4's claim that 'the seam is the story' is not established. I request a control experiment that fixes re-roll frequency or total pair-update budget while varying only seam revisits, for example a Sequential CBQ variant with periodic re-rolls but no second seam pass, or an ICBQ variant with an equal number of extra non-seam pair refinements.","section":"§5.4, Tables 3–5; Algorithm 1 step 12; Algorithm 2 step 1"},{"comment":"The theoretical centerpiece, the gamma^S factor in Theorem B.5 and Corollary B.8, depends on Assumption B.2: every actual two-block CBQ call must contract the per-sample midpoint residual uniformly by a constant gamma in (0,1), plus a fixed residual epsilon_sub. The paper itself states in Remark B.3 that rollback does not imply this certificate, and Appendix B.11 states that the empirically measured gamma is an aggregate diagnostic (mean 0.67, max 0.92) and 'is not, by itself, the uniform per-sample contraction constant required by Assumption B.2.' The proposed uniform estimator in Eq. (74) is not computed. Thus the theorem's prediction of geometric seam-amortized decay is not empirically supported in the uniform form needed; the theoretical claim is conditionally valid but its applicability to the reported systems remains open. I recommend either computing a uniform certificate on the actually visited finite sets (with an explicit epsilon_opt), or substantially reframing the theory section as a conditional bound and weakening the claim that the measured gamma diagnostics validate the amortization gain.","section":"Assumption B.2, Eq. (21); Remark B.3; Appendix B.11, Eq. (74)"},{"comment":"The informal theorem statement says the comparison is made 'with the same CBQ optimizer budget,' but ICBQ performs S additional pair refinements over Sequential CBQ, and Table 5 reports a mean wall-clock increase of 1.21x. The formal theorem does not require equal total budget, but the informal statement is misleading and obscures the confound that part of ICBQ's gain could simply be additional optimization passes on the same objective. Please either remove 'same CBQ optimizer budget' from the informal statement, define it as per-call per-pair budget, or explicitly discuss the extra compute as part of the method's cost. The equal-budget claim currently contradicts the provided runtime data.","section":"Theorem 4.1 and Table 5"},{"comment":"The GPTQ results show that ICBQ is not uniformly better than Sequential CBQ: at W3 on Mistral-7B ICBQ has PPL 8.86 versus 8.77 for Sequential CBQ, and there are several zero-shot task-level exceptions in Table 7. The text acknowledges this, but the framing in the abstract and conclusion that 'ICBQ reduces perplexity relative to the matched Sequential CBQ baseline' should be carefully qualified to the ternary DBF setting and to the aggregate perplexity comparisons. More importantly, the paper never compares against the original CBQ of Ding et al. or any other state-of-the-art block-wise PTQ method. Given that the claim is a schedule-level improvement, a comparison against the original multi-block CBQ window would establish practical relevance beyond the constructed matched baseline.","section":"§5.3, Table 2"}],"minor_comments":[{"comment":"The definition of S(L,K) is inconsistent: Eq. (11) uses a floor expression while the notation table and Theorem 4.1 / Theorem B.5 use S(L,K)=ceil(L/K)-1. These differ when L is not divisible by K, and the proof of Lemma B.10 relies on the ceiling version. Please make the definitions consistent throughout.","section":"§3.2, Eq. (11) and Notation table"},{"comment":"All reported perplexity and zero-shot numbers appear to come from a single run with no error bars or multiple seeds. Given that the effects in Table 3 for K=8 vs K=4 are sometimes small (e.g., Llama-2-7B 13.07 vs 12.5), reporting variance across seeds or at least stating single-seed status would help assess the stability of the monotonic trend.","section":"§5.1 and Tables 1–4"},{"comment":"The header 'The seam is the story' overstates what the ablation can show, given the confounds described in the first major comment. A more neutral title such as 'Chunk-size and schedule ablations' would better match the evidence.","section":"§5.4 header"},{"comment":"The Limitations paragraph mentions the block-locality and language-model scope, but it does not mention the unverified Assumption B.2 or the confounded seam ablation. Since these are the two main caveats on the paper's central claims, they should be acknowledged in the limitations.","section":"Limitations (Section 7)"},{"comment":"The wall-clock comparison reports 11 models, including four small models not present in the main seven-model tables (TinyLlama, Qwen3-0.6B, Gemma-2-2B, Gemma-3-1B). It would be helpful to state whether the main-text claims or only the runtime overhead claim use these additional models.","section":"Appendix C.3, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical core: the interleaved schedule consistently improves over the matched Sequential CBQ baseline in a demanding ternary setting, and the ablation trend with K is striking. My major concerns are (1) the empirical attribution of the gain specifically to seam revisits is not yet established because re-rolls and compute budget vary together with seam count; and (2) the theoretical gamma^S prediction relies on a per-sample contraction assumption that the authors themselves state is not certified by their measurements. Both are load-bearing for the paper's central narrative, but both are fixable within the manuscript's scope by additional controlled experiments and by either computing a uniform certificate or clearly relegating the theory to a conditional analysis. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is simple and worth taking seriously: interleave a short CBQ sweep at chunk boundaries so each seam pair gets refined twice, and formalize the benefit as a gamma^S factor on the propagated error term. That is a genuine scheduling twist on CBQ, and the paper is honest that the theory is bound-to-bound rather than a certificate on realized perplexity. The experiments are also consistently in one direction — ICBQ beats the matched Sequential CBQ baseline on all seven models with ternary DBF, rescues Qwen3-8B from divergence, and shows a clear monotone trend as K shrinks. That is real evidence, not noise.\n\nBut the stress-test note has it right, and this is the load-bearing weakness. ICBQ differs from the baseline in three ways at once: seam revisits, periodic activation re-rolls, and a larger total number of pair-updates (measured at about 1.21x wall-clock). The chunk-size ablation in Table 3 varies all three simultaneously, so the monotone improvement with smaller K is exactly what you would expect from more re-rolls or more compute alone. The paper's Section 5.4 claim that \"the seam is the story\" is not established, because no experiment holds re-roll frequency or refinement budget fixed while varying only seam revisits. Given that the paper's contribution is specifically the seam mechanism, this is a central confound that needs a dedicated experiment.\n\nThe theory has a softer but related gap: Assumption B.2 requires uniform per-sample contraction of every two-block subproblem, and the reported gamma is only an aggregate diagnostic on Llama-2-7B. The authors acknowledge this in Remark B.3 and Appendix B.11, which is to their credit, but it means the amortization-gain prediction is conditional on an unverified premise. Minor issues include no error bars or multiple seeds, no comparison to the original CBQ or other SOTA PTQ methods, and no released code or data.\n\nBottom line: this is a serious, well-structured paper with a plausible mechanism and an honest writeup. The confound is disqualifying only for the current claim, not for the idea itself. A good referee should send it to major revision and ask for a disentangling experiment — e.g., a sequential baseline with the same periodic re-rolls but no seam pass, or a version with matched total pair-update budget. That is the kind of feedback that would turn a conditional result into a convincing one.","headline":"Neat schedule idea and an honest conditional bound, but the empirical seam attribution is confounded by re-rolls and extra compute, so the headline claim needs disentangling.","tokens_in":39004,"tokens_out":1796,"would_cite":false,"duration_ms":17113,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Interleaving the quantization schedule so each chunk-boundary pair is refined twice shrinks propagated activation error by a $\\gamma^S$ factor and improves extreme low-bit perplexity.","keywords":["post-training quantization","cross-block quantization","large language models","low-bit quantization","ternary quantization","scheduling","error propagation","ICBQ"],"falsifier":"Log the pre- and post-pass midpoint residual per calibration sample at every CBQ call on a real model and compute the smallest uniform $\\gamma$ satisfying $r_{\\mathrm{out}} \\le \\gamma r_{\\mathrm{in}} + \\epsilon_{\\mathrm{sub}}$. If any sample's post-pass residual exceeds the affine bound with $\\gamma \\ge 1$, or if no finite uniform $\\gamma<1$ fits, the depth-wise bound of Theorem 4.1 does not hold.","tokens_in":37898,"feed_emoji":"🧵","tokens_out":4308,"duration_ms":36106,"temperature":0.7,"pith_summary":"This paper argues that a scheduling change—not a new objective or architecture—can make extreme low-bit quantization of large language models substantially more stable. In cross-block post-training quantization, errors made at early block pairs propagate through the network; the paper's method, Interleaved Cross-Block Quantization (ICBQ), re-refines each boundary pair between consecutive chunks of blocks, so each \"seam\" is optimized twice. Under stated local contraction and smoothness assumptions, the paper proves a depth-wise bound in which seam revisits multiply the propagated error term by $\\gamma^S$ while the additive residual stays bounded independently of depth. In experiments, ICBQ improves ternary-quantization perplexity relative to the matched sequential baseline on all seven main models, often turning divergent perplexity into finite values, and it also helps with 3-bit and 2-bit GPTQ.","feed_headline":"Re-refining chunk seams rescues 2-bit LLM quantization","feed_subtitle":"Interleaving one extra CBQ pass at each boundary pair lowers perplexity under ternary and 2-bit weight-only schemes.","key_machinery":"The carrying object is the chunk seam and its two-pass visit pattern. The outer driver quantizes blocks left-to-right in chunks of size $K$; at each chunk boundary it runs a two-block cross-block quantization (CBQ) refinement over a window that includes the last pair of the finished chunk and the first pair of the next chunk. Because the last pair of chunk $c$ is also the first pair of chunk $c+1$, that seam pair receives two CBQ passes. The paper isolates the seam effect in Lemma B.11 and shows the second pass reuses the same prefix, giving an extra contraction factor $\\gamma$ on the propagated term. The memory cost is one stored layer-0 activation buffer, independent of depth and chunk size.","core_discovery":"The central claim is that revisiting chunk seams—the boundary pairs between consecutive chunks of quantized Transformer blocks—converts a single left-to-right sweep into a schedule with repeated refinement at those seams, and this repetition alone contracts the propagated activation mismatch. The paper models the mismatch passed from one two-block window to the next, $m_i = \\|\\hat{X}_{i+1} - X^\\star_{i+1}\\|_F$, and shows each non-seam CBQ step obeys $m_i \\le \\tau m_{i-1} + (1+\\gamma)\\epsilon_{\\mathrm{sub}}$, while a seam visit gives an extra factor $\\gamma$: $m_i \\le \\gamma\\tau m_{i-1} + (1+\\gamma+\\gamma^2)\\epsilon_{\\mathrm{sub}}$. Chaining these recurrences yields Theorem 4.1: with $S = \\lceil L/K\\rceil -1$ seams, the interleaved bound is $\\gamma^S$ times the sequential propagated term plus a residual with a depth-independent constant. The paper reports that this schedule-level change lowers perplexity with ternary DBF on all seven evaluated models, and also applies to GPTQ at 3-bit and 2-bit precision.","pith_inferences":["The same seam-revisit principle should apply to larger cross-block windows or other prefix-locality pipelines, since the theoretical argument only needs a contractive two-block subproblem and an exact re-roll.","A direct testable extension is to certify the uniform per-sample contraction constant by logging per-sample midpoint residuals; the paper leaves this for future work and only reports aggregate diagnostics.","If the schedule transfers beyond language models, as the paper leaves open, it would make extreme low-bit quantization a drop-in schedule change for other transformer families."],"forward_implications":["Smaller chunk sizes create more seams and should improve the bound; the paper's experiments show perplexity improves monotonically or near-monotonically as $K$ goes from $L$ to 8, 4, and 2.","ICBQ transfers the schedule gain to GPTQ at W3 and W2g128, achieving best or tied-best perplexity on five of seven models.","The extra computation is only the re-refinement passes and one constant-size activation buffer, with no depth- or chunk-dependent memory growth.","The schedule keeps the local two-block objective, so it can be composed with stronger inner quantizers or local objectives rather than replacing them."],"supporting_citations":[{"why":"CBQ, whose cross-block reconstruction principle the Sequential CBQ baseline instantiates and which the paper directly compares against.","marker":"Ding et al., 2025"},{"why":"GPTQ, the Hessian-aware inner quantizer used for the W3 and W2g128 transfer experiments.","marker":"Frantar et al., 2022"},{"why":"DBF, the double-binary factorization whose ternary extension is the paper's primary inner quantizer.","marker":"Boža and Macko, 2026"}],"fun_headline_variants":["Seam revisits shrink 2-bit LLM perplexity","Interleaved seams refine low-bit LLM quantization","Second pass at seams rescues fragile 2-bit LLMs","Interleaved Cross-Block Quantization stabilizes 2-bit LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every two-block CBQ subproblem must contract the per-sample midpoint residual by a uniform factor $\\gamma \\in (0,1)$ plus a fixed residual $\\epsilon_{\\mathrm{sub}}$; the paper notes rollback alone does not certify this and only measures an aggregate $\\gamma$ on one model.","fun_headline_variants_meta":{"raw":{"variants":["Seam revisits shrink 2-bit LLM perplexity","Interleaved seams refine low-bit LLM quantization","Second pass at seams rescues fragile 2-bit LLMs","Interleaved Cross-Block Quantization stabilizes 2-bit LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000321,"raw_usage":{"total_tokens":1838,"prompt_tokens":1004,"completion_tokens":834,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":762}},"tokens_in":620,"tokens_out":834,"duration_ms":8252,"temperature":1.0,"reasoning_tokens":762,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:23:52.598100+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the pre- and post-pass midpoint residual per calibration sample at every CBQ call on a real model and compute the smallest uniform $\\gamma$ satisfying $r_{\\mathrm{out}} \\le \\gamma r_{\\mathrm{in}} + \\epsilon_{\\mathrm{sub}}$. If any sample's post-pass residual exceeds the affine bound with $\\gamma \\ge 1$, or if no finite uniform $\\gamma<1$ fits, the depth-wise bound of Theorem 4.1 does not hold.","supporting_citations":[{"cited_title":"Addition is almost all you need: Compressing large language models with double binary factorization","cited_arxiv_id":null,"evidence_quote":"DBF, the double-binary factorization whose ternary extension is the paper's primary inner quantizer."}],"review_version":1}