REVIEW 7 cited by
SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have proven to be highly effective across various natural language processing tasks. However, their large number of parameters poses significant challenges for practical deployment. Pruning, a technique aimed at reducing the size and complexity of LLMs, offers a potential solution by removing redundant components from the network. Despite the promise of pruning, existing methods often struggle to achieve substantial end-to-end LLM inference speedup. In this paper, we introduce SLEB, a novel approach designed to streamline LLMs by eliminating redundant transformer blocks. We choose the transformer block as the fundamental unit for pruning, because LLMs exhibit block-level redundancy with high similarity between the outputs of neighboring blocks. This choice allows us to effectively enhance the processing speed of LLMs. Our experimental results demonstrate that SLEB outperforms previous LLM pruning methods in accelerating LLM inference while also maintaining superior perplexity and accuracy, making SLEB as a promising technique for enhancing the efficiency of LLMs. The code is available at: https://github.com/jiwonsong-dev/SLEB.
Forward citations
Cited by 7 Pith papers
-
Prune&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation
A magnitude-compensation step that rescales the weights before a pruned layer, combined with iterative pruning, consistently improves perplexity and question-answering accuracy of layer-pruned LLMs.
-
ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations
ProcrustesGPT searches for per-layer orthogonal rotations that make pretrained LLM weights fit Kronecker or GS structured matrices, cutting 14 to 36 percent of parameters without fine-tuning.
-
Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layers
A training-free layer pruning framework for large vision-language models, combining token importance scoring with subspace-compensated weight projection, preserves most accuracy while speeding inference.
-
DipSVD: Dual-importance Protected SVD for Efficient LLM Compression
DipSVD combines channel-weighted whitening with layer-wise compression ratios and reports better perplexity and accuracy than existing SVD-based LLM compression methods.
-
SlimLLM: Accurate Structured Pruning for Large Language Models
SlimLLM prunes LLMs by scoring whole attention heads and feed-forward channels with Pearson similarity and PCA-based measures, then restores accuracy with per-dimension linear regression and non-uniform layer ratios.
-
Numerical Simulation for General Relativistic Magnetohydrodynamics in Dynamic Spacetimes
An abstract describing a novel spectral GRMHD solver is accompanied by an unrelated full text on VLM block skipping, rendering the claims unsubstantiated.
-
SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling
SkipGPT learns per-token, per-module routers that dynamically skip attention and MLP blocks, then uses LoRA to recover most of the original model's quality.
Discussion (0). Continue with ORCID to comment.