Pith. sign in

REVIEW 10 cited by

LLM-Pruner: On the Structural Pruning of Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.11627 v3 pith:ZXVAOPAE submitted 2023-05-19 cs.CL

Xinyin Ma , Gongfan Fang , Xinchao Wang This is my paper · ORCID
classification cs.CL
keywords languagellm-prunermodelsgenerationllmstrainingcapabilitiescompression
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. However, such impressive capability typically comes with a substantial model size, which presents significant challenges in both the deployment, inference, and training stages. With LLM being a general-purpose task solver, we explore its compression in a task-agnostic manner, which aims to preserve the multi-task solving and language generation ability of the original LLM. One challenge to achieving this is the enormous size of the training corpus of LLM, which makes both data transfer and model post-training over-burdensome. Thus, we tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures based on gradient information, maximally preserving the majority of the LLM's functionality. To this end, the performance of pruned models can be efficiently recovered through tuning techniques, LoRA, in merely 3 hours, requiring only 50K data. We validate the LLM-Pruner on three LLMs, including LLaMA, Vicuna, and ChatGLM, and demonstrate that the compressed models still exhibit satisfactory capabilities in zero-shot classification and generation. The code is available at: https://github.com/horseee/LLM-Pruner

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 74 citations worldwide. Full citation record

  1. Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Unsigned differential activations locate a few GLU-MLP neurons whose zeroing surgically destabilizes demographic bias while retaining ~99.5% of measured capabilities.

  2. BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A ReLU-routed MoE with chunk-level sparsity training objectives and custom kernels combining activation sparsity with speculative decoding achieves over 70% 8-token chunk sparsity and up to 3.67x end-side speedup.

  3. The Case for Instance-Optimized LLMs in OLAP Databases

    cs.DB 2025-07 conditional novelty 6.0 of 10

    IOLM-DB compresses a general-purpose LLM on a per-query basis to speed up LLM-invoking OLAP queries, reporting throughput gains up to 3.31x and smaller models in a three-workload prototype.

  4. Projectable Models: One-Shot Generation of Small Specialized Transformers from Large Ones

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A task-conditioned projection operator converts a large transformer's weights into a smaller task-specialized transformer that outperforms same-size universal conditional models.

  5. A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs

    cs.LG 2026-02 reject novelty 5.0 of 10

    Replicate-and-Quantize copies the busiest MoE expert as a quantized duplicate and compresses the least important expert, lowering a new Load-Imbalance Score by up to 1.4x while accuracy varies by roughly -1.2 to +3.0 points.

  6. Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

    cs.CL 2025-10 conditional novelty 5.0 of 10

    A single per-layer error tolerance, instead of a per-layer rank budget, improves zero-shot low-rank compression of LLMs and VLMs empirically, but the theorem that this uniform tolerance is Pareto-optimal is proven onl...

  7. A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering

    cs.LG 2025-07 reject novelty 5.0 of 10

    A clustering method for LLM activation patterns achieves up to 79% centroid precision and a best perplexity of 12.49, but the reported perplexity assumes 100% accurate cluster selection and no predictor is built.

  8. CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering

    cs.CV 2025-08 reject novelty 4.0 of 10

    A specialist classifier feeding a pruned VLM with knowledge-graph grounding reports 82.1% diagnostic accuracy on a 39-image dermatology test set, about 18 percentage points above a fine-tuned VLM baseline.

  9. Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression

    cs.AI 2025-07 conditional novelty 4.0 of 10

    A TPE-guided search over layer-wise pruning ratios and KV cache bit-widths compresses LLaVA-1.5 7B/13B with small accuracy loss, outperforming Wanda and SparseGPT on most tested benchmarks.

  10. CE-LSLM: Efficient Large-Small Language Model Inference and Communication via Cloud-Edge Collaboration

    cs.NI 2025-05 reject novelty 4.0 of 10

    CE-LSLM reports lower edge inference latency by reusing cloud LLM KV caches in edge SLMs, but the cross-model reuse mechanism is asserted, not demonstrated.

Pith tools