Pith. sign in

REVIEW 5 major objections 5 minor 59 references

GeLaCo: An Evolutionary Approach to Layer Compression

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Evolutionary search over layer merges finds LLM compressions that match or beat structured-pruning baselines.

desk verdict First EA for layer-collapse compression with a real Pareto front, slightly oversold in the abstract and reliant on an imperfectly validated similarity proxy. read the letter →

arxiv 2507.10059 v1 pith:6GHSUNFG submitted 2025-07-14 cs.CL

David Ponce , Thierry Etchegoyhen , Javier Del Ser This is my paper
classification cs.CL
keywords layercompressionevolutionaryalgorithmstructuredpruningmerginglargelanguagemodelsmulti-objectiveoptimizationParetofrontmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GeLaCo tries to show that layer-collapse compression of large language models is better framed as an evolutionary search than as a one-off empirical configuration. The paper's bet is that a cheap fitness signal, average cosine similarity of attention, feed-forward, and hidden-state modules between the original and compressed model, can guide a population of merge plans to solutions that match or beat structured pruning baselines like LaCo, LLM-Pruner, and SliceGPT. If that bet holds, compression becomes an automated optimization problem rather than a manual tuning exercise, and the same search can also optimize compression ratio and quality jointly. The paper reports the strongest evidence on Llama-3.1-8B, where GeLaCo gets the highest average scores across all tested compression ratios, and it constructs the first Pareto frontiers showing where quality collapses with layer removal. That matters because it gives practitioners a concrete way to choose a compression level for a target quality budget, and it shows the frontier is model-size dependent.

What carries the argument

The mechanism is differential weight merging of consecutive layers, following the LaCo update $\theta^*_l = \theta_l + \sum_{k=1}^{m}(\theta_{l+k} - \theta_l)$, which collapses a block of layers into its base layer by adding successive parameter differences. Around that operation, GeLaCo builds a module-wise similarity fitness function: for each of 64 calibration sentences it averages cosine similarities between original and compressed attention projections (Q, K, V, output), feed-forward projections (gate, up, down), and final hidden states, then averages over sentences. The evolutionary algorithm uses integer encoding of merge operations with an on/off flag, crossover and polynomial mutation, a repair mechanism that forces solutions to a target compression ratio, and a cache that reuses fitness scores for identical operation sets; the multi-objective variant uses NSGA-II to retain a spread of nondominated solutions. That machinery lets the search explore merge configurations without enumerating the exponentially large space, and the cache is what makes an 8B-model search finish in roughly 30 minutes on a single GPU.

What would settle it

Hold out a benchmark suite not used in the search, and for the Llama-3.1-70B Pareto front around 0.5 compression, rank solutions by the module-wise similarity score and by held-out accuracy; if the Spearman correlation between the two rankings is near zero, the fitness proxy is demonstrably not tracking quality at that compression level.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLM compression can be cast as an evolutionary search over layer-merge operations and that this search finds better or equally good solutions than hand-tuned structured pruning. Concretely, GeLaCo encodes candidate merge plans as integer triples (base layer, end layer, activation flag), scores each plan by the average cosine similarity of attention projections, feed-forward projections, and final hidden states between the original and the compressed model, and evolves the population under a genetic algorithm for fixed compression targets or under NSGA-II for the two objectives of compression and quality. On the model family tested, the paper reports that GeLaCo achieves the highest average benchmark scores across all compression ratios on Llama-3.1-8B, ties or leads on Llama-2 7B and 13B against LaCo, LLM-Pruner and SliceGPT, produces Pareto fronts that dominate LaCo's solutions, and after post-training with up to 10B tokens matches or exceeds the evolutionary pruning baseline DarwinLM on a comparable-size compressed model. For instruction-tuned models, GeLaCo maintains a median instruction-following score of 4 out of 5 up to 50% compression after recovery fine-tuning, with quality falling off sharply at 75% compression.

Load-bearing premise

The search is steered by cosine similarity between original and compressed modules, averaged over only 64 Wikipedia sentences; if that proxy stops tracking true task quality, the evolutionary search will keep picking merge plans that look similar but perform poorly.

Editorial extensions

If this is right

  • Compression targets that previously required per-model empirical tuning can be reached automatically: GeLaCo repairs any candidate to the exact target ratio, so practitioners can ask for, say, 25% or 50% layer reduction and get a configuration directly.
  • The first Pareto frontiers over compression ratio and quality give a concrete map of where capability collapses, and the paper shows the frontier differs by model scale: Llama-3.1-70B quality drops to near-random around 0.5 compression while smaller models degrade more gradually.
  • Post-training data volume transfers directly into quality: the paper reports steadily improving benchmark scores as fine-tuning tokens grow from 2.5B to 10B, so compressed models can be treated as regular pretraining checkpoints.
  • Instruction-tuned models survive moderate compression after recovery fine-tuning, with median scores of 4 on Just-Eval up to 50% compression and a drop at 75%, establishing a practical operating range for deployment.
  • Because module-wise similarity outperforms KL divergence and perplexity as a search objective on the tested setup, activation-space similarity is a viable cheap guide for evolutionary compression, not just a diagnostic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own 70B result suggests the similarity proxy saturates while real quality falls; a natural extension is to add a small held-out task probe as a second fitness signal or stopping rule, since the proxy alone may overestimate usable compression for very large models.
  • The search only merges consecutive layers, but the encoding would generalize to non-consecutive or partial merges; testing that extension would show whether the reported gains come from the evolutionary search or from the layer-merge operation itself.
  • Because the fitness uses only 64 Wikipedia sentences, the selected merges may be domain-biased; a testable variation is to rerun the search with calibration text from different domains and compare whether the Pareto front and the chosen layer merges change.
  • The caching mechanism trades population diversity for speed, and the paper notes convergence toward cached configurations; adding a diversity-promoting term could extend the explored frontier, at the cost of the 30-minute runtime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. GeLaCo is an evolutionary approach to LLM compression by layer merging. It encodes candidate merge plans as integer triples, evolves them with a genetic algorithm or NSGA-II, and ranks solutions using a module-wise cosine-similarity fitness computed on 64 Wikipedia sentences, averaging attention, feed-forward, and hidden-state similarities. The paper reports single-objective results on Llama-2 7B/13B and Llama-3.1 8B against LaCo, LLM-Pruner, and SliceGPT; bi-objective Pareto-front results on Llama-2 7B/13B and Llama-3.1 8B/70B; post-training results on Llama-3.1 8B against DarwinLM; and instruction-following results on Llama-3.1 8B Instruct. The central claim is that GeLaCo rapidly finds compression solutions that match or improve over state-of-the-art alternatives and produces the first Pareto frontier along compression and quality axes.

Significance. If the central claim holds, GeLaCo would be a useful contribution: it extends evolutionary compression to layer merging, supports multi-objective search, reports three-run averages with standard deviations, and evaluates on both base and instruction-tuned models with external benchmarks that are not used in the fitness function. The clearest supporting evidence is on Llama-3.1 8B, where GeLaCo obtains higher average benchmark scores than LaCo and LLM-Pruner at the tested ratios. However, the paper's own results on Llama-2 models show only ties or mixed outcomes, and the paper reports a sharp divergence between similarity fitness and downstream quality for Llama-3.1 70B. The significance of the contribution therefore depends on strengthening the validation of the fitness proxy and on tempering or statistically supporting the blanket outperformance claims.

major comments (5)
  1. [Abstract; §5.2, Table 2] The abstract's claim of 'outperforming state-of-the-art alternatives' is contradicted by Table 2 in several configurations. On Llama-2 7B, GeLaCo ties with LLM-Pruner at 0.125 compression (0.58 vs 0.580 average) and is not the best at 0.281 (LaCo 0.491, LLM-Pruner 0.489, GeLaCo 0.487) or at 0.406 (LaCo 0.443, GeLaCo 0.442). On Llama-2 13B, LaCo leads at 0.150 (0.627 vs 0.62) and at 0.425 (0.484 vs 0.458). The 'consistently highest average scores across all compression ratios' statement in Section 5.2 therefore holds only for Llama-3.1 8B in this table. Please either revise the global claims to reflect the mixed results or provide a statistical test, such as paired significance testing across the benchmark suite, that supports the stronger claim.
  2. [§5.2, Table 2] Baseline methods are reported as point estimates without variance, while GeLaCo results include standard deviations from three runs. For example, on Llama-3.1 8B at 0.125 compression, GeLaCo averages 0.642±0.003 versus LLM-Pruner's 0.605, a gap of 0.037; without error bars on LLM-Pruner it is unclear whether this difference is meaningful. Because the headline claim of consistent superiority relies on these comparisons, the paper should report multiple baseline runs or otherwise quantify baseline variability, or explicitly discuss the limitation that only GeLaCo has error bars.
  3. [§5.3, Figure 3] The Pareto front is optimized in the objective space of compression ratio and module-wise similarity, but the abstract and conclusions describe it as a frontier along 'compression and quality' axes. Figure 3 overlays HellaSwag scores, but HellaSwag is not one of the objectives used in the search. Consequently, the claimed Pareto dominance over LaCo is established only for the similarity proxy, not for downstream task quality. Please either re-label the axes consistently as (compression ratio, module similarity) or add an analysis that reports Pareto dominance or rank correlation in terms of held-out downstream quality.
  4. [§5.3, Limitations] The paper itself reports that for Llama-3.1 70B, 'quality scores drop to near-random levels at approximately 0.5 compression ratio, while the similarity metric remains relatively stable.' This is direct evidence that the fitness function does not track downstream capability in a regime that the paper claims to explore. Table 1 provides only a single comparison of fitness functions on Llama-2 7B at 0.281 compression; it does not establish that the module-wise similarity metric ranks candidate merge plans by downstream quality across models and ratios. Please add a validation of the fitness proxy, for example Spearman correlation between fitness and held-out benchmark averages over a sample of candidate plans for several models, or substantially restrict the claims about rapid discovery of state-of-the-art solutions.
  5. [§3.2, §4] The fitness function uses only 64 sentences sampled from English Wikipedia, and the paper provides no sensitivity analysis for this calibration set size or for the particular sample. Since the evolutionary search selects merge plans by maximizing similarity on this small set, the stability of the selected plans and of downstream scores with respect to different calibration samples should be investigated. A short experiment varying the calibration set (e.g., 32, 64, 128 sentences, or multiple random draws) would indicate whether the reported results are robust or depend on this free parameter.
minor comments (5)
  1. [§3.1, Multiple Objectives] The sentence 'the dual optimization of compression ratio and similarity preservation and compression ratio' contains a duplicated objective and should be rephrased, for example as 'the dual optimization of compression ratio and similarity preservation.'
  2. [§5.1, Table 1] In Table 1, the perplexity-optimized model achieves lower perplexity (13.730) than the module-similarity-optimized model (15.174); the text should acknowledge that the proposed fitness function improves average downstream task performance but does not optimize language-model perplexity.
  3. [§5.2, footnote 4] The compression ratios achieved by LaCo differ from those in the original LaCo paper, but the exact LaCo hyperparameters, layer counts, and merge thresholds used in this replication are not reported; please provide these details in an appendix or supplementary material to support reproducibility.
  4. [§5.3, Figure 3] The inverted scales for similarity and HellaSwag in Figure 3 are difficult to read; adding explicit axis titles and arrows or labels indicating which direction is better for each metric would improve clarity.
  5. [§2.2, Related Work] The claim of establishing 'the first Pareto front approximations' should be checked against the cited evolutionary compression works, particularly EvoPress and DarwinLM, and the comparison should be made explicit if any of those works also consider multi-objective formulations.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: downstream benchmarks are external, the fitness proxy is not fitted to them, and the only self-citation (JMetalPy) is non-load-bearing.

full rationale

The derivation chain is self-contained with respect to circularity. GeLaCo's search is driven by a module-wise cosine-similarity fitness computed on 64 Wikipedia sentences (Sections 3.2 and 4); the reported benchmark scores (BoolQ, PIQA, HellaSwag, etc.) come from the lm-evaluation-harness and are never used as fitness terms or fitted parameters. The post-training stage uses Fineweb-Edu and LaMini, not the evaluation benchmarks. The only self-citation is the JMetalPy framework (Benítez-Hidalgo et al., 2019), whose author list overlaps with the paper; this is a standard GA/NSGA-II implementation and is not load-bearing for the compression claims. The acknowledged divergence between similarity and downstream quality, e.g. 'quality scores drop to near-random levels at approximately 0.5 compression ratio, while the similarity metric remains relatively stable' (Section 5.3), and the limitation that the fitness function may not fully capture model capability on downstream tasks (Conclusions and Limitations) are validity threats to the proxy, not evidence that any benchmark result reduces by construction to the fitness. Likewise, the abstract's 'quality axes' description of the multi-objective Pareto front is clarified in Section 5.3, where the dual objectives are 'compression ratio and module-wise similarity' and HellaSwag is used only as an external evaluation overlay; overclaiming the axis label is a reporting issue, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central method rests on the assumption that representation similarity is a valid proxy for capability, and on the borrowed differential weight-merging formula. Hand-chosen settings (equal component weights, 64-sentence calibration set) affect the fitness landscape. No new entities are postulated.

free parameters (2)
  • Module-wise similarity weights = equal: 1/3 attention, 1/3 feed-forward, 1/3 hidden state (not fitted)
    Hand-chosen equal weighting of the three fitness components; the limitations section acknowledges that alternative weighting could change the set of selected solutions.
  • Calibration set size = 64 sentences
    A random subset of English Wikipedia; the same subset is used for LaCo, but the subset is not released, so fitness estimates depend on this hand-picked sample.
assumptions (4)
  • domain assumption Differential weight merging (Eq. 1) produces valid collapsed layers
    Adopted from LaCo; the paper does not verify that this merge rule preserves function, it only optimizes over its application.
  • domain assumption Cosine similarity between modules is a reliable proxy for downstream task capability
    Central to the fitness function; the authors observe divergence between similarity and HellaSwag for Llama-3.1-70B and flag it as a limitation.
  • domain assumption 64 Wikipedia sentences are representative for estimating similarity
    The fitness is estimated on a tiny sample; no analysis of variance over calibration sets is provided.
  • domain assumption Post-training on Fineweb-Edu or LaMini recovers capability lost during compression
    Post-training results depend on these datasets being suitable for recovery; only one baseline (DarwinLM) is compared.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeLaCo: An Evolutionary Approach to Layer Compression." pith.science (2026). https://pith.science/paper/6GHSUNFG

@misc{pith2026250710059,
  author       = {Pith},
  title        = {Pith review of: GeLaCo: An Evolutionary Approach to Layer Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6GHSUNFG}},
  note         = {Machine review of arXiv:2507.10059}
}
read the original abstract

Large Language Models (LLM) have achieved remarkable performance across a large number of tasks, but face critical deployment and usage barriers due to substantial computational requirements. Model compression methods, which aim to reduce model size while preserving its capacity, are an important means to mitigate these issues. Promising approaches along these lines, such as structured pruning, typically require costly empirical search for optimal variants and may run the risk of ignoring better solutions. In this work we introduce GeLaCo, an evolutionary approach to LLM compression via layer collapse. Our approach supports an efficient exploration of the compression solution space via population-based search and a module-wise similarity fitness function capturing attention, feed-forward, and hidden state representations. GeLaCo also supports both single and multi-objective evolutionary compression search, establishing the first Pareto frontier along compression and quality axes. We evaluate GeLaCo solutions via both perplexity-based and generative evaluations over foundational and instruction-tuned models, outperforming state-of-the-art alternatives.

Figures

Figures reproduced from arXiv: 2507.10059 by the authors.

Figure 1
Figure 1. Layer mapping for similarity evaluation be [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Mean Pareto front approximation for dual objective optimization across different Llama models showing [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Violin plots showing the distribution of [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 25 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. SliceGPT : Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024

    arXiv 2024

  4. [4]

    Antonio Ben \' tez-Hidalgo, Antonio J Nebro, Jos \'e Garc \' a-Nieto, Izaskun Oregi, and Javier Del Ser. 2019. jMetalPy : A Python framework for multi-objective optimization with metaheuristics. Swarm and Evolutionary Computation, 51:100598

    work page 2019

  5. [5]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. PIQA : Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7432--7439

    work page 2020

  6. [6]

    Tom Brown et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877--1901

    work page 2020

  7. [7]

    Zouying Cao, Yifei Yang, and Hai Zhao. 2024. Head-wise shareable attention for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2555--2571

    work page 2024

  8. [8]

    Yupeng Chang et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1--45

    work page 2024

Show all 59 references
  1. [9]

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. 2024. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10558--10578

    2024

  2. [10]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A ...

    2019 doi

  3. [11]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try ARC , the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457

    2018 arXiv

  4. [12]

    Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2023. SpQR : A sparse-quantized representation for near-lossless LLM weight compression. arXiv preprint arXiv:2306.03078

    2023 arXiv

  5. [13]

    Elias Frantar and Dan Alistarh. 2023. SparseGPT : Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323--10337. PMLR

    2023

  6. [14]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  7. [15]

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2022. A survey of quantization methods for efficient neural network inference. In Low-power Computer Vision , pages 291--326. Chapman and Hall/CRC

    2022

  8. [16]

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International Journal of Computer Vision, 129(6):1789--1819

    2021

  9. [17]

    Aaron Grattafiori et al. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  10. [18]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2023. MiniLLM : Knowledge distillation of large language models. arXiv preprint arXiv:2306.08543

    2023 arXiv

  11. [19]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  12. [20]

    Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. BiLLM : pushing the limit of post-training quantization for LLMs . In International Conference on Machine Learning, pages 20023--20042

    2024

  13. [21]

    Weizhong Huang, Yuxin Zhang, Xiawu Zheng, Fei Chao, and Rongrong Ji. 2025. Towards efficient automatic self-pruning of large language models. arXiv preprint arXiv:2502.14413

    2025 arXiv

  14. [22]

    Renren Jin, Jiangcun Du, Wuwei Huang, Wei Liu, Jian Luan, Bin Wang, and Deyi Xiong. 2024. A comprehensive evaluation of quantization strategies for large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 12186--12215

    2024

  15. [23]

    Jangho Kim, Yash Bhalgat, Jinwon Lee, Chirag Patel, and Nojun Kwak. 2019. QKD : Quantization-aware knowledge distillation. arXiv preprint arXiv:1911.12491

    2019 arXiv

  16. [24]

    Jangho Kim, Simyung Chang, and Nojun Kwak. 2021. PQK : model compression via pruning, quantization, and knowledge distillation. arXiv preprint arXiv:2106.14681

    2021 arXiv

  17. [25]

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.248 Prometheus 2: An open source language model specialized in evaluating other la...

    2024 doi

  18. [26]

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2023. The unlocking spell on base LLMs : Rethinking alignment via in-context learning. arXiv preprint arXiv:2312.01552

    2023 arXiv

  19. [27]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ : Activation-aware weight quantization for on-device LLM compression and acceleration. Proceedings of Machine Learning and Systems, 6:87--100

    2024

  20. [28]

    Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. LogiQA : A challenge dataset for machine reading comprehension with logical reasoning. arXiv preprint arXiv:2007.08124

    2020 arXiv

  21. [29]

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023. LLM-QAT : Data-free quantization aware training for large language models. arXiv preprint arXiv:2305.17888

    2023 arXiv

  22. [30]

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2024 a . LLM-QAT : Data-free quantization aware training for large language models. In Findings of the Association for Computational ...

    2024

  23. [31]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024 b . SpinQuant: LLM quantization with learned rotations . arXiv preprint arXiv:2405.16406

    2024 arXiv

  24. [32]

    Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. 2024. https://doi.org/10.57967/hf/2497 Fineweb-edu: the finest collection of educational content

    2024 doi

  25. [33]

    Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024. The era of 1-bit LLMs : All large language models are in 1.58 bits. arXiv preprint arXiv:2402.17764

    2024 arXiv

  26. [34]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. LLM-pruner : On the structural pruning of large language models. Advances in Neural Information Processing Systems, 36:21702--21720

    2023

  27. [35]

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. ShortGPT : Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853

    2024 arXiv

  28. [36]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. https://doi.org/10.18653/v1/D18-1260 Can a suit of armor conduct electricity? a new dataset for open book question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language P...

    2018 doi

  29. [37]

    Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, and Pavlo Molchanov. 2024. Compact language models via pruning and knowledge distillation. Advances in Neural Information Process...

    2024

  30. [38]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8)

    2019

  31. [39]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. WinoGrande : An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  32. [40]

    Oliver Sieberling, Denis Kuznedelev, Eldar Kurtic, and Dan Alistarh. 2024. EvoPress : Towards optimal dynamic model compression via evolutionary search. arXiv preprint arXiv:2410.14649

    2024 arXiv

  33. [41]

    Alemi, and Andrew G

    Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A. Alemi, and Andrew G. Wilson. 2021. Does knowledge distillation really work? Advances in Neural Information Processing Systems, 34:6906--6919

    2021

  34. [42]

    Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Zhiqiang Shen, and Dan Alistarh. 2025. Darwinlm: Evolutionary structured pruning of large language models. arXiv preprint arXiv:2502.07780

    2025 arXiv

  35. [43]

    Hugo Touvron et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  36. [44]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  37. [45]

    Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. BitNet : Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453

    2023 arXiv

  38. [46]

    Ziheng Wang, Jeremy Wohlwend, and Tao Lei. 2020. Structured pruning of large language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6151--6162

    2020

  39. [47]

    Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. arXiv preprint arXiv:1707.06209

    2017 arXiv

  40. [48]

    Edward WD Whittaker and Bhiksha Raj. 2001. Quantization-based language model compression. In INTERSPEECH, pages 33--36

    2001

  41. [49]

    Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji. 2023. http://arxiv.org/abs/2304.14402 Lamini-lm: A diverse herd of distilled models from large-scale instructions . CoRR, abs/2304.14402

    2023 arXiv

  42. [50]

    Shangyu Wu, Hongchao Du, Ying Xiong, Shuai Chen, Tei-wei Kuo, Nan Guan, and Chun Jason Xue. 2025. EvoP : Robust LLM inference via evolutionary pruning. arXiv preprint arXiv:2502.14910

    2025 arXiv

  43. [51]

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. 2023. Sheared Llama : Accelerating language model pre-training via structured pruning. arXiv preprint arXiv:2310.06694

    2023 arXiv

  44. [52]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  45. [53]

    Canwen Xu and Julian McAuley. 2023. A survey on model compression and acceleration for pretrained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 10566--10575

    2023

  46. [54]

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024 a . A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116

    2024 arXiv

  47. [55]

    Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. 2024 b . OneBit : Towards extremely low-bit large language models. arXiv preprint arXiv:2402.11295

    2024 arXiv

  48. [56]

    Chuanpeng Yang, Yao Zhu, Wang Lu, Yidong Wang, Qian Chen, Chenlong Gao, Bingjie Yan, and Yiqiang Chen. 2024 a . Survey on knowledge distillation for large language models: methods, evaluation, and application. ACM Transactions on Intelligent Systems and Technology

    2024

  49. [57]

    Yifei Yang, Zouying Cao, and Hai Zhao. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.372 L a C o: Large language model pruning via layer collapse . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6401--6417, Miami, Florida, USA. Assoc...

    2024 doi

  50. [58]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

    2019 arXiv

  51. [59]

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2024. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12:1556--1577

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.