Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Truncating activations rather than weights makes SVD compression competitive with pruning: Dobi-SVD cuts LLaMA-7B to 40% parameters at WikiText2 perplexity 9.95 with a 12.4x speedup.

desk verdict Empirically strong and genuinely novel SVD compression; the 'theoretically optimal' weight-update proof has a real gap, so treat that claim as a heuristic. read the letter →

arxiv 2502.02723 v1 pith:TF2O6VET submitted 2025-02-04 cs.LG

Qinsi Wang , Jinghan Ke , Masayoshi Tomizuka , Yiran Chen , Kurt Keutzer , Chenfeng Xu This is my paper · ORCID
classification cs.LG
keywords modelcompressionsingularvaluedecompositionlow-rankapproximationlargelanguagemodelsactivationtruncationdifferentiableincrementalPCAmixed-precisionquantization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dobi-SVD claims that SVD-based LLM compression has been acting on the wrong object: rather than truncating weight matrices, or rescaling weights to respect activations, one should truncate the activations themselves — the product $xW$ — and then reconstruct a weight matrix that reproduces the truncated activations. On that principle the paper builds three mechanisms: a differentiable procedure that learns each layer's truncation position with only 224 trainable parameters, a memory-efficient IPCA weight reconstruction derived from the Eckart-Young-Mirsky theorem, and a quantized storage remap that removes a structural information-loss floor of standard SVD. The reported payoff is that LLaMA-7B compressed to a 0.4 parameter ratio reaches 9.95 perplexity on WikiText2, far below earlier SVD methods such as SVD-LLM at 53.74 and competitive with structured pruning methods that need fine-tuning, with a 12.4x inference speedup on a 12GB GPU. The same pipeline is applied to the LLaVA vision-language model and the OpenVLA vision-language-action model, with near-lossless results reported. The sympathetic reading is that if activation truncation is genuinely optimal, SVD becomes a third viable track for LLM compression, orthogonal to quantization and pruning.

What carries the argument

The machinery is the truncated activation $A_k = A V_A G_k V_A^T$ (with $G_k$ the rank-$k$ mask) and the reconstructed weight $fW = W V_A G_k V_A^T$ that reproduces it, computed by Incremental Principal Component Analysis (IPCA), the incremental variant of PCA that processes matrices one at a time to avoid storing all activations at once. Three supporting mechanisms carry the pipeline: a smooth truncation function $T(\sigma_i) = \sigma_i[0.5\cdot\tanh(\beta(k-i)) + 0.5]$ that makes the discrete rank $k$ differentiable and trainable; a stabilized SVD backward pass that replaces the vanishing denominators $1/(\sigma_i^2 - \sigma_j^2)$ with a geometric-series Taylor approximation to prevent gradient explosions; and a remapping storage scheme that stores $k\cdot\max(m,n)$ rather than $k\cdot(m+n)$ entries by 8-bit quantizing and concatenating rows of the $U$ and $V$ factors, exploiting their approximately Gaussian distribution to keep quantization error small.

What would settle it

A decisive check is to compute $fW$ for a single layer of a small language model in three ways — the paper's IPCA solution, the unconstrained least-squares minimizer of the true objective in Eq. (5), and the plain rank-$k$ SVD of $W$ — and compare the actual objective value and the downstream WikiText2 perplexity at ratio 0.4. If the IPCA solution does not minimize the true objective even when the unconstrained minimizer does, the theoretical-optimality claim is refuted; if activation truncation still wins empirically, only the proof, not the principle, is falsified. A second check: the paper reports that without remapping, differentiable truncation gives 46.18 perplexity versus 58.02 for uniform truncation at ratio 0.4, so a replication that finds no such gap would falsify the differentiable-truncation claim.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the optimal rank-$k$ replacement for a weight matrix $W$ in a transformer is not the truncated SVD of $W$ but the matrix $fW = W V_A G_k V_A^T$ obtained from the SVD of the activation matrix $A = xW$, where $V_A$ holds the right singular vectors of $A$ and $G_k$ is the diagonal mask that keeps the top $k$ singular directions. By the Eckart-Young-Mirsky theorem, truncating $A$ at rank $k$ yields the closest rank-$k$ matrix to $A$ in Frobenius norm, and the paper argues at both the module level and the model level that this makes direct activation truncation strictly better than weight truncation. The weight reconstruction then becomes the problem of finding the rank-$k$ matrix closest to the set of projected weights $\{W V_{A_i} G_k V_{A_i}^T\}$ over a set of input samples, which the paper solves with Incremental Principal Component Analysis for memory efficiency and claims to be theoretically optimal. The paper further identifies a long-overlooked limitation of SVD-based compression — the storage ratio $k(m+n)/(mn)$ is injective in $k$, so for square matrices even a storage ratio of 1 forces half the singular values to be discarded — and fixes it by storing $k\cdot\max(m,n)$ entries with mixed-precision quantization of the near-Gaussian $U$ and $V$ factors, turning the truncation-to-ratio map into a bijection. The reported consequence is that SVD-based compression becomes competitive with structured pruning: 9.95 perplexity on WikiText2 for LLaMA-7B at 40% of its parameters, versus 53.74 for SVD-LLM at the same ratio.

Load-bearing premise

The load-bearing premise is that the derivation of the theoretically optimal weight update is valid: it factors the activation matrix $x_i$ out of the Frobenius norm in Eq. (5), a step that is only justified for scalars or isometries, and it converts the inequality in Eq. (6) into the optimization objective without proving that the bound is tight, so if either step fails the IPCA-computed $fW$ is not the minimizer of the stated objective.

Editorial extensions

If this is right

  • SVD becomes a viable third track for LLM compression: at ratio 0.4, Dobi-SVD reports WikiText2 perplexity 9.95 on LLaMA-7B, a relative improvement of about 78% over SVD-LLM, and on LLaMA-3.1-8B at ratio 0.6 the average commonsense-reasoning accuracy drop is 8.7%, versus 33% for LLM-Pruner and SliceGPT.
  • The compressed model delivers hardware wins precisely where memory is the bottleneck: LLaMA-7B at ratio 0.4 fits in 6.8GB and generates 25.97 tokens per second on a 12GB Titan Xp, a 12.4x speedup over the CPU-swapping baseline, with up to 1.75x on an A100.
  • SVD and quantization stack rather than compete: combining Dobi-SVD with GPTQ-4bit brings LLaMA-7B to 3.4GB at ratio 0.6 with 9.97 WikiText2 perplexity.
  • The truncation-position search is cheap by LLM standards: 224 trainable scalars, 256 training samples, and about 8 GPU hours on LLaMA-7B, with no weight fine-tuning and no extra data at reconstruction time.
  • The method transfers beyond text: compressing the LLM block of LLaVA-v1.5-7B keeps the average VQA accuracy drop at 8.3% at ratio 0.4 while Pope-random accuracy rises to 89.8%, and OpenVLA-7B keeps 92.97% action accuracy at ratio 0.4, speeding up 17% and dropping to 5.2GB.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the remapping scheme is a statement about SVD storage in general, not about LLMs specifically, so any SVD-based compression store (images, embeddings, scientific matrices) could adopt the $k\cdot\max(m,n)$ mixed-precision packing; the 'half the singular values must go' critique applies to a storage convention, not to SVD itself.
  • Editorial inference: if activation truncation is genuinely the right target, it suggests a general 'activation-first' compression template — decide which activation directions matter, then solve the weight reconstruction — and Dobi-SVD's differentiable $k$ is one instantiation; cheaper sensitivity heuristics might approximate the same truncation profiles with less training.
  • Editorial inference: a testable extension the paper leaves open is stability of the learned truncation positions across random seeds and sample draws; because training uses only 224 scalars and 256 samples, re-running the search several times would show whether the per-layer profiles reflect a structural property of the model or a noise artifact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Dobi-SVD, an SVD-based LLM compression method built on three components: differentiable optimization of per-matrix truncation ranks k via a smoothed tanh truncation with stabilized SVD backpropagation; a weight-update step that uses incremental PCA (IPCA) on right-singular vectors of activations to compute a rank-k proxy weight matrix; and a mixed-precision remapping that packs quantized left and right factors into an m×k storage format so that the compression ratio becomes k/rank(W). The authors argue that directly truncating activations A=xW is theoretically and empirically superior to truncating weights or to activation-aware weight scaling. Experiments on LLaMA-7B, LLaMA2-7B, LLaMA3.1-8B, LLaMA-13B, LLaVA-v1.5, and OpenVLA report perplexity improvements over prior SVD methods (e.g., WikiText2 PPL around 9.95 at a 0.4 parameter ratio on LLaMA-7B), competitive or better behavior relative to structured pruning, and substantial inference speedups on both A100 and Titan Xp hardware.

Significance. If the empirical results hold, this is a practically significant contribution: it would make post-training SVD-based compression competitive with structured pruning at low parameter ratios, using only 256 calibration samples and 224 trainable parameters without fine-tuning. The paper is also genuinely useful in its extensive evaluation across models, tasks, quantization combinations, and VLM/VLA settings, and it reports enough hyperparameters to reproduce the main pipeline. The claimed theoretical optimality, however, is not established: the derivation of the 'theoretically optimal weight update' contains a clear algebraic error and an unjustified relaxation, and the model-level 'proof' in Appendix A.10 is an empirical observation. The lasting value of the paper therefore depends on either correcting the derivations or reframing the method as an empirically motivated heuristic with strong experimental support.

major comments (4)
  1. [Section 3.2, Eq. (5)] Equation (5) is not a valid algebraic reduction. The activation x_i is a matrix, so ||x_i (W V_i G_k V_i^T - fW)||_F cannot be rewritten as x_i ||W V_i G_k V_i^T - fW||_F; the right-hand side is a matrix multiplied by a scalar norm and is not even a scalar objective. The correct formulation is a weighted least-squares problem min_{fW} Σ_i ||x_i (W V_i G_k V_i^T - fW)||_F^2, whose minimizer depends on the Gram matrices x_i^T x_i. Algorithm 2's unweighted IPCA on the projected weights therefore does not compute the minimizer of the stated objective, and the additional restriction fW = W V V^T is not shown to be without loss of generality for a rank-k minimizer. Answer 2's claim of a 'theoretically optimal rank-k weight matrix' is thus unsupported.
  2. [Appendix A.4.1, Eq. (6)] Equation (6) replaces the optimization objective with an upper bound. Submultiplicativity gives Σ_i ||W(V_i V_i^T - V V^T)||_F^2 ≤ ||W||_F^2 Σ_i ||V_i V_i^T - V V^T||_F^2. Minimizing the right-hand side need not minimize the left-hand side, and the paper supplies no tightness argument or constant-ratio argument. Consequently the PCA objective in Eqs. (7)-(8) is not shown to be equivalent to the Eq. (5) criterion, and the IPCA solution cannot be claimed optimal for that criterion.
  3. [Section 3.1 (Algorithm 1) vs Section 3.2 (Algorithm 2)] The truncation positions k are trained on the smooth activation-truncated model, whose reconstructed activations are eA = U_A Σ_A G_k V_A^T, but the deployed model uses fW = W V G_k V^T computed via IPCA. These are different transformations of the weight matrix, and no argument is given that ranks which minimize loss under the first proxy also minimize loss under the deployed model. Table 16 shows that training helps relative to uniform ranks, but it does not establish optimality for the deployed model. The language of 'optimal truncation position' should either be backed by a connecting argument or softened.
  4. [Appendix A.10] The section is presented as a proof that activation truncation dominates weight truncation at the model level, but Eqs. (10)-(11) only rewrite first-order loss changes. The inequality ΔL_A ≤ ΔL_W is never derived; it is instead verified empirically in Fig. 11 on five selected layers. This does not support the claim in Section 2.3 that 'we prove (A.10)'. If the paper wishes to claim theoretical optimality of activation truncation, the proof needs to be completed; otherwise this should be presented plainly as an empirical observation.
minor comments (6)
  1. [Abstract / Section 4.1 / Table 2 / Conclusion] The reported WikiText2 perplexity at a 0.4 parameter ratio on LLaMA-7B is inconsistent across the paper: the abstract says 9.07, Section 4.1 says 9.70, Table 2 and the conclusion say 9.95. Section 4.1 also quotes the ASVD perplexity as 43,104 while Table 2 reports 57,057. Please reconcile these numbers.
  2. [Appendix A.3] The baseline list cites 'SVD-LLM (Rajpurkar, 2016)', which should be Wang et al., 2024, and the list ends with an incomplete 'Self-' entry. Several reference entries are duplicated (An et al. 2024a/2024b and Sakaguchi et al. 2021a/2021b) and should be merged.
  3. [Algorithm 2, line 6] The incremental centering step is not described correctly: the running mean µ should be updated before centering the new observation, and the formula V_i - (µ + (V_i - µ)/i) does not define a consistent update of µ across iterations. Please rewrite this step with explicit mean-update notation.
  4. [Section 2.1] The text states that U and V are 'the right and left singular vector matrices, respectively'; this should read 'left and right', since U is the left singular-vector matrix and V is the right singular-vector matrix.
  5. [Algorithm 3 / Section 3.3] The description of the packing step says 'first n rows of V^T_fW,k (both n×k)', which is dimensionally inconsistent because V^T is k×n. Please rewrite the remapping procedure with explicit shapes for UΣ and V^T so that the claimed m×k storage format is unambiguous.
  6. [Figure 3 / Section 4.2.1] The figure labels '256 Train Sample' and '16 Train Sample' are inconsistent with the text's discussion of batch sizes 256 and 16; clarify whether the number of training samples or the batch size is being varied.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: reported gains are not forced by construction; the main weaknesses are mathematical relaxations, not circular reductions.

  1. renaming known result [Section 2.3 (and Appendix A.10, Eq. A.10-A.11)]
    "At the module level, Proposition 2 shows that directly truncating activations gives Ak, the optimal k-rank approximation of A. At the model level, we prove (A.10) that directly truncating activations better minimizes training loss than truncating weights."

    The module-level 'optimality' is the Eckart-Young-Mirsky theorem restated: Ak is defined as the best rank-k approximation under the Frobenius norm, so calling activation truncation 'the optimal approach' is equivalent to the theorem's own conclusion rather than being derived from the LLM compression objective. The model-level extension is not a proof either; Appendix A.10 concludes 'we conclude that ... holds for the majority of activations' from the empirical Figure 11 comparison. This is a minor definitional/renaming flavor, not a forced prediction: the independent content is the empirical comparison and the concrete fW/IPCA construction.

full rationale

The paper is largely self-contained against external, standard results: EYM, SVD backpropagation formulas, and IPCA from Weng et al. (2003) are external and are not replaced by a self-citation chain. The truncation positions k are trained on 256 calibration samples, and the reported Wikitext2 PPL is an evaluation of the resulting compressed model; this is calibration, not a circular prediction forced by construction. The weight update does contain a serious gap: Eq. (5) factors the non-scalar activation matrix x_i out of the Frobenius norm, which is invalid for general x_i, and Eq. (6) replaces the objective with an untight upper bound before applying PCA. However, these are correctness and tightness problems in an approximation argument, not cases where the output is equivalent to the input by definition. The only mild circularity is the rhetorical identification of 'optimal SVD compression' with the EYM best-rank-k approximation: that is a known theorem renamed as a 'new perspective', and the model-level extension is empirically concluded rather than proved. Since no load-bearing self-citation or fitted-parameter-renamed-as-prediction step was found, the circularity score is 1.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard linear algebra (EYM, orthogonality), two domain assumptions about activation statistics and singular-vector distributions, and one ad hoc assumption that an upper-bound optimization is equivalent to the true objective. The free parameters are the per-matrix truncation ranks plus a few hand-set hyperparameters; there are no invented physical entities.

free parameters (5)
  • Truncation rank k (per matrix) = 224 trainable scalars for LLaMA-7B (per-layer values not reported)
    Learned on 256 WikiText2 samples through the multi-objective loss; directly controls compression ratio and performance.
  • beta (tanh smoothness) = 10
    Hand-set in Appendix A.3; controls the steepness of the soft truncation function.
  • gamma (singular value floor) = 1e-10
    Hand-set in Appendix A.3; prevents division by near-zero singular values during SVD backpropagation.
  • K (Taylor expansion terms) = 10
    Hand-set in Appendix A.3; truncation order for the geometric series used in gradient stabilization.
  • Training sample count and epochs = 256 samples, 320 epochs, batch size 32
    Chosen by hand for the calibration set; affects the fitted k values.
assumptions (5)
  • standard math Eckart-Young-Mirsky theorem
    Used in Section 2.3 to conclude that the truncated SVD of an activation matrix is the optimal rank-k approximation at module level.
  • standard math Orthogonality of singular vector matrices U and V
    Invoked in Proposition 3 and used in Eq (3) to rewrite A_k = A V_A G_k V_A^T.
  • domain assumption Gaussian distribution of singular vector entries
    Assumed in Section 3.3 to justify 8-bit quantization of U and V; supported empirically in Appendix A.7.1 for specific layers of LLaMA-7B.
  • domain assumption Activations from the uncompressed model are representative for the compressed model
    Used throughout to collect activation statistics and compute weight updates without iterative re-estimation after earlier layers are compressed.
  • ad hoc to paper The inequality in Eq (6) is a valid proxy for the original objective
    Appendix A.4.1 replaces the actual minimization objective with an upper bound and treats the solution of the bound as the optimal weight update; tightness is not shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives." pith.science (2026). https://pith.science/paper/TF2O6VET

@misc{pith2026250202723,
  author       = {Pith},
  title        = {Pith review of: Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TF2O6VET}},
  note         = {Machine review of arXiv:2502.02723}
}
read the original abstract

We provide a new LLM-compression solution via SVD, unlocking new possibilities for LLM compression beyond quantization and pruning. We point out that the optimal use of SVD lies in truncating activations, rather than merely using activations as an optimization distance. Building on this principle, we address three critical challenges in SVD-based LLM compression: including (1) How can we determine the optimal activation truncation position for each weight matrix in LLMs? (2) How can we efficiently reconstruct the weight matrices based on truncated activations? (3) How can we address the inherent "injection" nature that results in the information loss of the SVD? We propose Dobi-SVD, which establishes a new, principled approach to SVD-based LLM compression.

Figures

Figures reproduced from arXiv: 2502.02723 by the authors.

Figure 1
Figure 1. Overview framework of Dobi-SVD: 1-3: Differentiable Truncation Position Training. By applying parameter renormalization for continuous rank ratio selection and using Taylor expansion to prevent gradient explosion, our method enables robust and adaptive optimization of truncation positions. 4: Weight Update. Using IPCA, we sequentially extract and optimally update weight matrix features. 5: Remapping. We resolve a lo… view at source ↗
Figure 3
Figure 3. (Left) Performance Comparison of different training methods on LLaMA-7b. For activation [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Tokens/sec of original LLaMA-7B and its compressed version by Dobi-SVD under 40%, [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Data distribution of Attention Q matrix of Llama-7b layer 20 [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 7
Figure 7. Figure 7: The decrease in training loss and ppl over the training epochs for the Llama-7b model when trained with Wiki￾text2. 0 5 10 15 20 25 30 Layer Index (i) 340 360 380 400 420 440 460 Gamma Attention K Matrix_repara 0 5 10 15 20 25 30 Layer Index (i) 360 380 400 420 440 Gam…
Figure 8
Figure 8. Figure 8: k changes over time for different layers. Experiments were performed on the Wikitext2 dataset and the LLaMA-7b with a target compression ratio of 0.4. The model was trained for 20 epochs (colors range from yellow to purple). The red line indicates the initial gamma val…
Figure 9
Figure 9. Figure 9: k changes over time for different layers. Experiments were performed on the Wikitext2 dataset and the LLaMA-7b with a target compression ratio of 0.2 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: k changes over time for different layers. Experiments were performed on the Wikitext2 dataset and the LLaMA-7b with a target compression ratio of 0.6. Rank Adjustment Percentage on Llama-2-7b (Dobi 0.4) PPL Degradation Perc entage on Wikitext2 0% 0% 0.024% 0.739% 0.12…
Figure 11
Figure 11. Figure 11: Performance comparison of directly truncating activations and truncating weights. We [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Performant Unified GPU Kernels for Portable Singular Value Computation Across Hardware and Precision

    cs.DC 2025-08 conditional novelty 6.0 of 10

    A unified Julia implementation of two-stage QR SVD achieves near-cuSOLVER performance across four GPU vendors and three precisions, including firsts for Apple Metal and half precision.

  2. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Loss-aware rank allocation plus residual-stream output correction yields substantially lower WikiText-2 perplexity than prior SVD LLM compressors at 60% compression.

  3. PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpoint

    cs.LG 2025-09 conditional novelty 3.0 of 10

    PHLoRA extracts LoRA-compatible adapters from full-rank fine-tuned models via truncated SVD of the weight delta, matching full-rank performance on several benchmarks with no gradients or training data.

Reference graph

Works this paper leans on

65 extracted references · 24 canonical work pages · cited by 3 Pith papers

  1. [1]

    Online embedding compression for text classification using low rank matrix factorization

    Anish Acharya, Rahul Goel, Angeliki Metallinou, and Inderjit Dhillon. Online embedding compression for text classification using low rank matrix factorization. In Proceedings of the aaai conference on artificial intelligence, volume 33, pp.\ 6196--6203, 2019

    work page 2019

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Mathqa: Towards interpretable math word problem solving with operation-based formalisms

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319, 2019

    arXiv 1905

  4. [4]

    Fluctuation-based adaptive structured pruning for large language models

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 10865--10873, 2024 a

    work page 2024

  5. [5]

    Fluctuation-based adaptive structured pruning for large language models

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 10865--10873, 2024 b

    work page 2024

  6. [6]

    Slicegpt: Compress large language models by deleting rows and columns

    Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024, 2024

    arXiv 2024

  7. [7]

    Polycystic kidney disease

    Carsten Bergmann, Lisa M Guay-Woodford, Peter C Harris, Shigeo Horie, Dorien JM Peters, and Vicente E Torres. Polycystic kidney disease. Nature reviews Disease primers, 4 0 (1): 0 50, 2018

    work page 2018

  8. [8]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 7432--7439, 2020

    2020

Show all 65 references
  1. [9]

    Compression of facial images using the k-svd algorithm

    Ori Bryt and Michael Elad. Compression of facial images using the k-svd algorithm. Journal of Visual Communication and Image Representation, 19 0 (4): 0 270--282, 2008

    2008

  2. [10]

    Model compression

    Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.\ 535--541, 2006

    2006

  3. [11]

    Adabert: Task-adaptive bert compression with differentiable neural architecture search

    Daoyuan Chen, Yaliang Li, Minghui Qiu, Zhen Wang, Bofang Li, Bolin Ding, Hongbo Deng, Jun Huang, Wei Lin, and Jingren Zhou. Adabert: Task-adaptive bert compression with differentiable neural architecture search. arXiv preprint arXiv:2001.04246, 2020

    2001 arXiv

  4. [12]

    Groupreduce: Block-wise low-rank approximation for neural language model shrinking

    Patrick Chen, Si Si, Yang Li, Ciprian Chelba, and Cho-Jui Hsieh. Groupreduce: Block-wise low-rank approximation for neural language model shrinking. Advances in Neural Information Processing Systems, 31, 2018

    2018

  5. [13]

    A survey of model compression and acceleration for deep neural networks

    Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282, 2017

    2017 arXiv

  6. [14]

    A comprehensive survey on model compression and acceleration

    Tejalal Choudhary, Vipul Mishra, Anurag Goswami, and Jagannathan Sarangapani. A comprehensive survey on model compression and acceleration. Artificial Intelligence Review, 53: 0 5113--5155, 2020

    2020

  7. [15]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  8. [16]

    Exploiting linear structure within convolutional networks for efficient evaluation

    Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. Exploiting linear structure within convolutional networks for efficient evaluation. Advances in neural information processing systems, 27, 2014

    2014

  9. [17]

    Everybody prune now: Structured pruning of llms with only forward passes

    Lucio Dery, Steven Kolawole, Jean-Fran c ois Kagy, Virginia Smith, Graham Neubig, and Ameet Talwalkar. Everybody prune now: Structured pruning of llms with only forward passes. arXiv preprint arXiv:2402.05406, 2024

    2024

  10. [18]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024

    2024

  11. [19]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pp.\ 10323--10337. PMLR, 2023

    2023

  12. [20]

    Optq: Accurate quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Optq: Accurate quantization for generative pre-trained transformers. In The Eleventh International Conference on Learning Representations, 2022 a

    2022

  13. [21]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022 b

    2022 arXiv

  14. [22]

    A framework for few-shot language model evaluation

    Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, et al. A framework for few-shot language model evaluation. Version v0. 0.1. Sept, 10: 0 8--9, 2021

    2021

  15. [23]

    An efficient svd-based method for image denoising

    Qiang Guo, Caiming Zhang, Yunfeng Zhang, and Hui Liu. An efficient svd-based method for image denoising. IEEE transactions on Circuits and Systems for Video Technology, 26 0 (5): 0 868--880, 2015

    2015

  16. [24]

    Dynabert: Dynamic bert with adaptive width and depth

    Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. Dynabert: Dynamic bert with adaptive width and depth. Advances in Neural Information Processing Systems, 33: 0 9782--9793, 2020

    2020

  17. [25]

    Language model compression with weighted low-rank factorization

    Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. Language model compression with weighted low-rank factorization. arXiv preprint arXiv:2207.00112, 2022

    2022 arXiv

  18. [26]

    Speeding up convolutional neural networks with low rank expansions

    Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014

    2014 arXiv

  19. [27]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla...

    2024 arXiv

  20. [28]

    Squeezellm: Dense-and-sparse quantization

    Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Mahoney, and Kurt Keutzer. Squeezellm: Dense-and-sparse quantization. arXiv preprint arXiv:2306.07629, 2023

    2023 arXiv

  21. [29]

    The singular value decomposition: Its computation and some applications

    Virginia Klema and Alan Laub. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control, 25 0 (2): 0 164--176, 1980

    1980

  22. [30]

    Speeding-up convolutional neural networks using fine-tuned cp-decomposition

    Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, Ivan Oseledets, and Victor Lempitsky. Speeding-up convolutional neural networks using fine-tuned cp-decomposition. arXiv preprint arXiv:1412.6553, 2014

    2014 arXiv

  23. [31]

    Mimo transmission over a time-varying channel using svd

    Guillaume Lebrun, Jason Gao, and Mike Faulkner. Mimo transmission over a time-varying channel using svd. IEEE Transactions on wireless Communications, 4 0 (2): 0 757--764, 2005

    2005

  24. [32]

    Lrq: Optimizing post-training quantization for large language models by learning low-rank weight-scaling matrices

    Jung Hyun Lee, Jeonghoon Kim, June Yong Yang, Se Jung Kwon, Eunho Yang, Kang Min Yoo, and Dongsoo Lee. Lrq: Optimizing post-training quantization for large language models by learning low-rank weight-scaling matrices. arXiv preprint arXiv:2407.11534, 2024

    2024 arXiv

  25. [33]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--...

    2024

  26. [34]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 26296--26306, 2024

    2024

  27. [35]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36: 0 21702--21720, 2023

    2023

  28. [36]

    Building a large annotated corpus of english: The penn treebank

    Mitch Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19 0 (2): 0 313--330, 1993

    1993

  29. [37]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016

    2016 arXiv

  30. [38]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    2018 arXiv

  31. [39]

    Acdc: A structured efficient linear layer

    Marcin Moczulski, Misha Denil, Jeremy Appleyard, and Nando de Freitas. Acdc: A structured efficient linear layer. arXiv preprint arXiv:1511.05946, 2015

    2015 arXiv

  32. [40]

    Image compression using svd

    HS Prasantha, HL Shashidhara, and KN Balasubramanya Murthy. Image compression using svd. In International conference on computational intelligence and multimedia applications (ICCIMA 2007), volume 3, pp.\ 143--145. IEEE, 2007

    2007

  33. [41]

    Squad: 100,000+ questions for machine comprehension of text

    P Rajpurkar. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016

    2016 arXiv

  34. [42]

    Matrix compression via randomized low rank and low precision factorization

    Rajarshi Saha, Varun Srivastava, and Mert Pilanci. Matrix compression via randomized low rank and low precision factorization. Advances in Neural Information Processing Systems, 36, 2023

    2023

  35. [43]

    Compressing large language models using low rank and low precision decomposition

    Rajarshi Saha, Naomi Sagan, Varun Srivastava, Andrea J Goldsmith, and Mert Pilanci. Compressing large language models using low rank and low precision decomposition. arXiv preprint arXiv:2405.18886, 2024

    2024 arXiv

  36. [44]

    Low-rank matrix factorization for deep neural network training with high-dimensional output targets

    Tara N Sainath, Brian Kingsbury, Vikas Sindhwani, Ebru Arisoy, and Bhuvana Ramabhadran. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. In 2013 IEEE international conference on acoustics, speech and signal processing, pp.\ 6...

    2013

  37. [45]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021 a

    2021

  38. [46]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021 b

    2021

  39. [47]

    The truth is in there: Improving reasoning in language models with layer-selective rank reduction

    Pratyusha Sharma, Jordan T Ash, and Dipendra Misra. The truth is in there: Improving reasoning in language models with layer-selective rank reduction. arXiv preprint arXiv:2312.13558, 2023

    2023 arXiv

  40. [48]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695, 2023

    2023 arXiv

  41. [49]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  42. [50]

    Pufferfish: Communication-efficient models at no extra cost

    Hongyi Wang, Saurabh Agarwal, and Dimitris Papailiopoulos. Pufferfish: Communication-efficient models at no extra cost. Proceedings of Machine Learning and Systems, 3: 0 365--386, 2021 a

    2021

  43. [51]

    Robust differentiable svd

    Wei Wang, Zheng Dang, Yinlin Hu, Pascal Fua, and Mathieu Salzmann. Robust differentiable svd. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 b

    2021

  44. [52]

    Svd-llm: Truncation-aware singular value decomposition for large language model compression

    Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. Svd-llm: Truncation-aware singular value decomposition for large language model compression. arXiv preprint arXiv:2403.07378, 2024

    2024 arXiv

  45. [53]

    Candid covariance-free incremental principal component analysis

    Juyang Weng, Yilu Zhang, and Wey-Shiuan Hwang. Candid covariance-free incremental principal component analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25 0 (8): 0 1034--1040, 2003

    2003

  46. [54]

    The devil is in the details --- W ikipedia , the free encyclopedia

    Wikipedia. The devil is in the details --- W ikipedia , the free encyclopedia. http://en.wikipedia.org/w/index.php?title=The\ [Online; accessed 24-November-2024]

    2024

  47. [55]

    Sheared llama: Accelerating language model pre-training via structured pruning

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared llama: Accelerating language model pre-training via structured pruning. arXiv preprint arXiv:2310.06694, 2023

    2023 arXiv

  48. [56]

    Asvd: Activation-aware singular value decomposition for compressing large language models

    Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, and Guangyu Sun. Asvd: Activation-aware singular value decomposition for compressing large language models. arXiv preprint arXiv:2312.05821, 2023

    2023 arXiv

  49. [57]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  50. [58]

    Lqer: Low-rank quantization error reconstruction for llms

    Cheng Zhang, Jianyi Cheng, George A Constantinides, and Yiren Zhao. Lqer: Low-rank quantization error reconstruction for llms. arXiv preprint arXiv:2402.02446, 2024

    2024 arXiv

  51. [59]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  52. [60]

    Accelerating very deep convolutional networks for classification and detection

    Xiangyu Zhang, Jianhua Zou, Kaiming He, and Jian Sun. Accelerating very deep convolutional networks for classification and detection. IEEE transactions on pattern analysis and machine intelligence, 38 0 (10): 0 1943--1955, 2015

    1943

  53. [61]

    A novel strategy for signal denoising using reweighted svd and its applications to weak fault feature enhancement of rotating machinery

    Ming Zhao and Xiaodong Jia. A novel strategy for signal denoising using reweighted svd and its applications to weak fault feature enhancement of rotating machinery. Mechanical Systems and Signal Processing, 94: 0 129--147, 2017

    2017

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  55. [63]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  56. [64]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  57. [65]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.