Pith. sign in

REVIEW 2 major objections 5 minor 9 references

ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Expanding inner rank lets ternary LLM quantization approach bf16 accuracy arbitrarily closely.

desk verdict Expanded-rank ternary decomposition is a genuine new dial for ternary PTQ, but 'accuracy' currently means reconstruction energy, and the sole end-to-end run trails Q4_K despite more bits. read the letter →

arxiv 2607.13511 v1 pith:RTNWZQID submitted 2026-07-15 cs.LG cs.AI

Chethan Reddy G.P This is my paper · ORCID
classification cs.LGcs.AI
keywords ternaryquantizationpost-trainingexpandedrankmatrixfactorizationsequentialdeflationLLMcompressionenergypreservationlow-bit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces ExTernD, a post-training factorization that writes each LLM weight matrix as the product of two ternary matrices (-1, 0, +1) with a real scale vector in between, and lets the inner dimension k exceed the matrix's full rank. The central claim is that adding components beyond full rank fits the residual left by previous components, so reconstruction error decreases monotonically and can be made as small as desired: in principle the decomposition can approach full-precision (bf16) accuracy, something no fixed-count ternary scheme can do. Practically, at matched reconstruction energy ExTernD uses about 5.2-5.5 effective bits per weight versus 4.5 bpw for the Q4_K baseline, and a full 4B model conversion at rank multiplier 3 reaches 10.10 wikitext-2 perplexity versus 9.78 for bf16, placing it near the Q4_K/Q5_K accuracy band. The significance is that ternary quantization stops being a lossy format with a hard ceiling and becomes a continuous accuracy/cost dial, tunable per matrix, which matters for multiplication-free inference hardware.

What carries the argument

The central object is the factorization A ≈ B diag(D) C with ternary factors B ∈ {-1,0,+1}^{m×k}, C ∈ {-1,0,+1}^{k×n} and a real scale vector D; its key parameter is the inner-rank multiplier μ = k/min(m,n), which acts as an accuracy dial. The argument is carried by sequential deflation: component i is fit to the residual R after previous components, and the optimal scale d_i = uᵀRv/(||u||²||v||²) makes the residual norm decrease monotonically. A batched block-ALS variant approximates this sequential deflation using decorrelated least-squares targets to avoid collapsing onto the dominant residual direction. The convergence proof adds a single-entry removal step (best residual entry) that gua

What would settle it

Run ExTernD with a large rank multiplier (μ≈10) on a 4B model weight matrix and measure energy; if it plateaus below the bf16 reference (e.g., below 99.99%) rather than continuing to increase, the practical 'arbitrarily close to bf16' claim fails for the implemented algorithm. Conversely, if a full conversion with all levers (sweeps, per-matrix μ, PPL-tuned τ/λ) does not close the perplexity gap to Q4_K at matched bits, then matrix-level energy parity is not transferring to task accuracy.

Watch

Extended reading notes

Core claim

ExTernD's core discovery is that a ternary factorization with an expanded inner rank — k = μ·min(m,n) with μ>1 — removes the fixed capacity ceiling of ternary quantization. Each added component is fit to the residual of its predecessors (sequential deflation), and with a least-squares scale the residual norm is monotone non-increasing; the paper proves it can be driven below any ε>0 by growing k, approaching bf16 accuracy. No fixed number of ternary planes has this property. Empirically, matching Q4_K's ~99.4–99.5% per-matrix energy requires only 5.2–5.5 effective bits per weight, and a full 4B conversion at μ=3 reaches 10.10 wikitext-2 perplexity vs 9.78 for bf16, near the Q4_K/Q5_K band.

Load-bearing premise

Per-matrix reconstruction energy (E = 1 − ‖A−Â‖²/‖A‖²) is treated as a proxy for model accuracy; the end-to-end perplexity test still shows a gap (10.10 vs 9.78 bf16), and the convergence proof assumes a best-entry step the implemented algorithm does not run.

Editorial extensions

If this is right

  • If the central claim holds, ternary LLM quantization no longer has a hard accuracy ceiling; any target error can be met by raising μ, with memory and compute growing linearly in μ.
  • Per-matrix μ and τ give a continuous accuracy/cost trade-off, so deployments can hit an exact bit/accuracy target instead of rounding to the next k-quant bit-width.
  • At matched effective bits, ExTernD matches the reconstruction energy of a 4-plane dense ternary stack while being ~56% sparse, which is friendlier to multiplication-free hardware.
  • A full 4B model can be converted post-training in ~20 minutes on a single GPU, producing coherent text near the Q4_K/Q5_K perplexity band, with identified levers (sweeps, per-matrix rank allocation, importance weighting) expected to close the remaining gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The proof of arbitrary accuracy relies on a best-single-entry augmentation that the implemented ALS does not run; whether the practical algorithm alone converges to arbitrarily small error remains to be demonstrated empirically.
  • If energy parity transfers to task performance across scales and tasks, the ~17–20% bpw gap to Q4_K is likely a multiplier (algorithmic) problem rather than a coding limit, since the packing is near the entropy floor.
  • The same sequential-deflation idea could in principle be applied to other structured formats, such as binary or low-bit factors, as a general way to remove fixed-capacity ceilings.
  • A short straight-through QAT pass on an ExTernD initialization is an obvious next test; if gradients through two chained ternary factors behave, it could recover the remaining perplexity gap faster than training a conventionally quantized model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper proposes ExTernD, a post-training factorization of a weight matrix A into A ≈ B diag(D) C, where B and C are ternary matrices and D is a real scale vector. The inner rank is set to k = μ min(m,n), deliberately larger than full rank, so that additional components correct the residual left by previous ones. The method is fitted with a greedy sequential ALS (Sec. 2.3), a batched block-ALS variant (Sec. 2.4), and an importance-weighted objective (Sec. 2.5). The paper claims monotonic residual decrease and convergence to any ε > 0 (Sec. 2.1, Appendix A), and reports per-matrix reconstruction energy matching Q4_K at 5.2–5.5 effective bpw (Table 2). One full Qwen3.5-4B conversion at μ=3 gives wikitext-2 PPL 10.099 versus 9.782 for bf16 and 10.015 for Q4_K pure (Table 3).

Significance. If the central claims held, ExTernD would remove the fixed-capacity ceiling of ternary post-training quantization and provide a continuous accuracy/cost trade-off for multiplication-free LLM inference. The least-squares monotonicity argument in Prop. 1 is correct and clean, and Prop. 2 is a valid construction showing that an augmented single-entry safeguard yields geometric convergence. The empirical observations about the decorrelated block-ALS target (Sec. 2.4) and the corrected V-step for imatrix weighting (Sec. 2.5) are also valuable. However, the convergence theorem does not apply to the implemented greedy ALS without the safeguard, and the empirical 'matching Q4_K accuracy' claim is based on reconstruction energy rather than task-level accuracy; the only end-to-end result trails Q4_K at a higher bitrate. These gaps are load-bearing for the paper's headline claims.

major comments (2)
  1. [Appendix A, Prop. 2; Sec. 2.3] Prop. 2 proves geometric convergence for an augmented algorithm that, at each step, chooses whichever is larger between the ALS update and the best single-entry removal. The algorithm actually implemented and described in Sec. 2.3 (and used in all experiments) does not perform this safeguard. The sentence 'In practice the safeguard never activates' is an unproved empirical assertion, not a proof. Consequently, the abstract's claim that ExTernD 'can be driven below any ε>0' and 'approaches bf16 accuracy arbitrarily closely' is not established for the method that is evaluated. To repair this, either add the safeguard to the implemented algorithm (e.g., as a fallback per iteration, checking whether a single-entry step gives a larger decrease), report its activation frequency, and re-run experiments with that inclusion; or explicitly state that the theorem applies to an augmented variant and
  2. [Abstract; Sec. 3.3; Table 3] The abstract labels the matched Q4_K reconstruction energy as 'per-matrix accuracy,' and Sec. 3.3 chooses μ to match Q4_K's 99.4–99.5% energy values. But Table 3, the paper's only end-to-end measurement, shows the μ=3 conversion at ~5.7 effective bpw reaching 10.099 wikitext-2 PPL, versus 10.015 for Q4_K pure at 4.51 bpw and 9.948 for Q5_K pure at 5.51 bpw. Thus at roughly 26% more bits than Q4_K, the decomposition is still about 0.8–1.5% worse in relative PPL. Reconstruction energy is therefore not a demonstrated proxy for task-level accuracy, and the proof in Appendix A bounds only Frobenius error, not PPL or any downstream metric. The paper itself concedes in Sec. 5 that 'end-to-end task accuracy is not yet measured.' The headline 'accuracy approaching any quantization level' is unsupported on the empirical side. Please either provide end-to-end results with the identified levers appl
minor comments (5)
  1. [Sec. 2.4, Eq. (3)] The transpose placement in V ← Tτ((U^T U + εI)^{-1} U^T R)^T is easy to misread. Clarify the intended dimensions, or rewrite the equation with explicit matrix shapes.
  2. [Table 3] The entries 'Q4 K M' and 'Q4 K pure' should be defined: the reader cannot tell what 'M' means from the table alone, nor whether 'bpw' is theoretical or measured after packing.
  3. [Sec. 5] The sentence 'our structural and representational claims against them are proven (Sec. 2.1)' is misleading: Sec. 2.1 contains no proof of a comparison against PTQTP or PT2-LLM, and Appendix A's proof concerns residual convergence, not relative comparison. Repoint the citation or temper the wording.
  4. [Sec. 3.6] The phrase 'This decisively proves' in the iso-BPW discussion is too strong for an energy-equality comparison on a handful of matrices; please soften to 'supports' or 'indicates'.
  5. [Sec. 4] Minor typo: 'TQ2 0' should likely be 'TQ2_0' or similar; also ensure the format name matches llama.cpp's actual tensor type notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ExTernD's residual-decrease guarantee is a consequence of its least-squares deflation, and the Q4_K comparisons use an external baseline; the energy-to-accuracy proxy gap is a correctness risk, not a circular step.

full rationale

The derivation chain is self-contained. Proposition 1's monotone residual decrease follows by substituting the optimal least-squares scale d_i into the residual norm identity; this is a mathematical consequence of the algorithm's own objective, not a fitted result renamed as a prediction. Proposition 2's epsilon-convergence is proven for an explicitly stated augmentation (best single-entry removal), and the paper does not hide that the implemented ALS omits it: 'In practice the safeguard never activates' is an empirical claim, not an assumed premise. No load-bearing self-citations or author-imported uniqueness theorems appear; the references to PTQTP, BitNet, GPTQ, etc. are external comparisons. The empirical headline 'matches Q4_K's per-matrix accuracy' is defined via reconstruction energy E = 1 - ||A - A_hat||^2/||A||^2, and mu is chosen per matrix to match Q4_K's energy, so the bpw figures in Table 2 are legitimate measurements of a fitted operating point rather than forced predictions. The paper's own Limitations section concedes that end-to-end task accuracy is not yet measured and that evaluation is perplexity-only; that is an unsupported proxy gap (correctness risk), not circularity. The statement that 'any target accuracy is reachable by construction' is an honestly labeled construction-based theorem, not an equation reducing to its inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method's theoretical guarantee is elementary and mostly sound, but it is carried by a tunable rank μ, a threshold τ, and the assumption that per-matrix energy is a proxy for model quality. No invented entities are needed.

free parameters (4)
  • μ (inner-rank multiplier) = per-matrix values 1.97–3.43; full conversion μ=3
    k=μ min(m,n) is the accuracy dial; chosen per matrix to match Q4_K energy in Sec 3.3 and fixed at 3 in Table 3. The headline 'target accuracy hit exactly' depends on tuning μ.
  • τ (sparsity threshold) = 0.7, 1.0, 2.0
    Eq. 2 sets the zero fraction in B and C; swept in Sec 3.2 and fixed in comparisons; directly sets bpw via Eq. 5.
  • λ (importance-weight interpolation) = 0
    Interpolates between uniform and imatrix objectives (Sec 2.5); all weighted results use λ=0.
  • Alternating iteration count / block size = 15 iterations; b=256 capped at min(m,n)/8
    Not fitted to data, but chosen hyperparameters that affect convergence and reported energy; not central to the capacity claim.
assumptions (4)
  • domain assumption Energy preservation (relative Frobenius norm) is a valid measure of model-level quantization accuracy.
    Abstract/Sec 3.3 use 'per-matrix accuracy' and 'matches Q4_K' based on energy values; Table 3 is the only PPL check and still trails Q5_K, so this assumption is load-bearing for the empirical headline.
  • ad hoc to paper The convergence proof applies to the implemented algorithm even though the safeguard step is not performed.
    Appendix A proves geometric convergence for an augmented update (best single-entry removal) that Sec 2.3's greedy ALS never runs; 'the safeguard never activates' is empirical, not proven.
  • domain assumption The mask+sign packing model (Eq. 5) correctly represents true storage cost of the factors, with real scales D absorbed into side data for free.
    Eq. 5 counts 2−sparsity bpw per element; the real scale vector D is k floats and is said to be absorbed into C's row scales in TQ20. This keeps reported bpw comparable to k-quant baselines.
  • domain assumption Per-input-channel second moments from llama.cpp imatrix approximate true activation importance.
    Sec 2.5 uses imatrix statistics h as weights; the corrected V-step relies on these statistics being representative of evaluation activations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level." pith.science (2026). https://pith.science/paper/RTNWZQID

@misc{pith2026260713511,
  author       = {Pith},
  title        = {Pith review of: ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RTNWZQID}},
  note         = {Machine review of arXiv:2607.13511}
}
abstract

We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix $A \in \mathbb{R}^{m \times n}$ into $A \approx B \mathrm{diag}(D) C$ with ternary factors $B \in \{-1,0,+1\}^{m \times k}$, $C \in \{-1,0,+1\}^{k \times n}$ and a real scale vector $D \in \mathbb{R}^k$. The inner rank $k = \mu \min(m,n)$ is deliberately expanded beyond full rank ($\mu > 1$), so that components past full rank correct the quantization error of earlier ones. We prove the residual decreases monotonically in $k$ and can be driven below any $\varepsilon > 0$: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do. Memory and compute scale continuously with $\mu$, and factor sparsity continuously with a threshold $\tau$, so an accuracy target is hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K's per-matrix accuracy at 5.2-5.5 effective bpw (5.1-5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at $\mu = 3$ reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%), placing it near the Q4_K/Q5_K accuracy band at ~5.7 effective bpw.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

9 extracted references · 5 linked inside Pith

  1. [1]

    H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y. Wu, F. Wei. BitNet: Scaling 1-bit Transformers for Large Language Models.arXiv:2310.11453, 2023

    arXiv 2023

  2. [2]

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, F. Wei. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.arXiv:2402.17764, 2024

    arXiv 2024

  3. [3]

    F. Li, B. Liu, X. Wang, B. Zhang, J. Yan. Ternary Weight Networks.arXiv:1605.04711, 2016. 8

    arXiv 2016

  4. [4]

    C. Zhu, S. Han, H. Mao, W. J. Dally. Trained Ternary Quantization.ICLR, 2017. arXiv:1612.01064

    arXiv 2017

  5. [5]

    H. Xiao, R. Yang, Q. Yang, W. Xu, Z. Li, Y. Su, Z. Liu, H. Yang, N. Wong. PTQTP: Post- Training Quantization to Trit-Planes for Large Language Models.arXiv:2509.16989, 2025

    arXiv 2025

  6. [6]

    X. Yan, W. Wu, Z. Wu, S. Yang, Z. Gao, M. Huang. PT2-LLM: Post-Training Ternarization for Large Language Models.arXiv:2510.03267, 2025

    arXiv 2025

  7. [7]

    Frantar, S

    E. Frantar, S. Ashkboos, T. Hoefler, D. Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers.arXiv:2210.17323, 2022

    arXiv 2022

  8. [8]

    J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, S. Han. A WQ: Activation-aware Weight Quantization for LLM Compression and Acceleration.MLSys,

Show all 9 references
  1. [9]

    Wang et al

    J. Wang et al. Bitnet.cpp: Efficient Inference Framework for 1-bit LLMs. https://github.com/microsoft/BitNet, 2024. 9

    2024

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.