Pith. sign in

REVIEW 3 major objections 5 minor 36 references

M-Wanda: Improving One-Shot Pruning for Multilingual LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read M-Wanda, a one-shot pruning method that adds language-aware activation statistics and correlation-based layerwise sparsity to Wanda, cuts average multilingual perplexity by 2–7% at 50% sparsity across six LLMs and improves all six…

desk verdict Genuinely novel pruning criterion with broad experiments, but the headline numbers are inflated by tuning hyperparameters on the same benchmark; held-out results still make it worth engaging. read the letter →

arxiv 2505.21171 v1 pith:BMAWQATV submitted 2025-05-27 cs.CL cs.AI

Rochelle Choenni , Ivan Titov This is my paper
classification cs.CLcs.AI
keywords multilingualpruningone-shotlargelanguagemodelslanguage-awareactivationstatisticslayerwisesparsityallocationcross-lingualsharingmodelcompressionretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard one-shot pruning methods such as Wanda are designed around English performance, and this paper shows that at moderate sparsity they quietly erode multilingual ability: at 50% sparsity, perplexity rises sharply, especially for underrepresented languages. The paper proposes M-Wanda, which reweights the pruning criterion with language-aware activation statistics and allocates sparsity across layers using a correlation-based measure of cross-lingual sharing. At 50% sparsity, M-Wanda reduces average Flores perplexity by 2–7% relative to Wanda across six multilingual LLMs, improves performance on all six zero-shot tasks for Llama-8B, and reduces average perplexity by 6% on 15 unseen languages. If the method is right, compression of multilingual models no longer has to sacrifice non-English languages disproportionately, and pruning evaluation should be multilingual by default.

What carries the argument

The mechanism is a modified per-weight importance score paired with a correlation-driven sparsity allocator. The score uses three language-aware activation statistics: a cross-lingual variance ratio $VAR = \frac{\mathrm{Var}_{\text{inter}}}{\frac{1}{|L|}\sum_{\ell}\mathrm{Var}_{\ell}^{\text{intra}}}$ that upweights neurons whose activation differs between languages but is stable within each language; an activation probability $P(I(|X_j| > \epsilon))$ that discounts high-variance neurons that rarely activate; and CWL sparsity, which takes Pearson correlations of mean activations per language and sublayer, adjusts inter-language correlation by intra-language stability, and prunes layers with higher cross-lingual similarity less aggressively. The full criterion is $S_{i,j} = (|W_{i,j}| \cdot AX_j) \cdot P(I(|X_j| > \epsilon))$, with $AX_j = \|X_j\|_2 + \lambda \cdot VAR$ and min-max normalization applied before adding the variance term. This combination is what lets M-Wanda keep both shared and specialized neurons after one-shot pruning.

What would settle it

Prune the same model with the CWL allocation reversed—giving the lowest sparsity to layers that CWL ranks lowest in inter-language correlation and the highest sparsity to layers it ranks highest—and compare Flores perplexity against M-Wanda; if the reversed allocation matches or beats M-Wanda, the correlation hypothesis is not doing the work and the gains come from the language-aware scoring terms instead.

Watch

Extended reading notes

Core claim

The paper's central claim is that multilingual degradation after one-shot pruning is not inevitable: it can be reduced by scoring weights with language-aware activation statistics and by pruning less aggressively in layers whose activation patterns are shared across languages. Concretely, M-Wanda scores weight $W_{i,j}$ as $S_{i,j} = (|W_{i,j}| \cdot AX_j) \cdot P(I(|X_j| > \epsilon))$, where $AX_j$ combines Wanda's activation norm $\|X_j\|_2$ with a normalized cross-lingual variance ratio, and the activation-probability term $P(I(|X_j| > \epsilon))$ filters out high-variance neurons that rarely fire. Layerwise sparsity is set by Correlation Weighted Layerwise (CWL) sparsity, which averages Pearson correlations of mean per-language activations across sublayers and assigns lower sparsity to layers with higher cross-lingual similarity. The paper reports consistent perplexity reductions on all calibration languages (2–7% at 50% sparsity), accuracy gains on all six downstream tasks for Llama-8B, generalization to 15 unseen languages (6% average perplexity decrease), and the same enhancement applied to RIA also improves it across all six models. It claims to be the first pruning method explicitly optimized for multilingual retention.

Load-bearing premise

The load-bearing premise is that layers whose mean activations are highly correlated across languages are the layers that support cross-lingual sharing, so pruning them less aggressively preserves multilingual ability; if inter-language activation correlation does not actually mark cross-lingual importance, the CWL allocation's gains could disappear.

Editorial extensions

If this is right

  • At 50% sparsity, the default in prior multilingual pruning work, Wanda already incurs substantial perplexity increases; M-Wanda's relative gains grow with sparsity, reaching a 52% average perplexity reduction over Wanda at 70% sparsity on Llama-8B.
  • M-Wanda improves on Wanda without retraining and with only minimal extra cost: it reuses the same activation statistics and keeps the same 128-sample calibration budget, at the price of a small hyperparameter search.
  • The language-aware scoring and CWL allocation are not tied to Wanda: applying them to RIA, a newer pruning criterion, yields consistent perplexity reductions across all six models.
  • Consistent gains on 15 languages not seen during calibration indicate that the method preserves language variance generally rather than memorizing calibration languages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension suggested by the typological-diversity analysis is to use CWL-style correlation scores to select which languages to calibrate on; the paper's data show small, typologically diverse subsets can beat the full 15-language set, so calibration-language choice might matter more than calibration size.
  • If the correlation hypothesis generalizes, the same language-aware variance and activation-probability terms could be added to structured N:M pruning or quantization-aware scoring; the paper only tests unstructured pruning and does not make this claim.
  • The larger gains on typologically distant languages (Arabic, Turkish, Vietnamese, Chinese, Korean, Japanese) suggest M-Wanda could double as a diagnostic for locating language-specific versus shared layers, though the paper does not pursue this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies how unstructured one-shot pruning degrades multilingual performance in LLMs. Using Wanda on six open-source models at multiple sparsity levels, the authors show that multilingual perplexity degrades sharply above 50% sparsity and that the degradation is largest for non-Indo-European languages. They then propose M-Wanda, which modifies the Wanda importance score by adding a normalized cross-lingual activation variance term and an activation-probability mask, and replaces uniform layerwise sparsity with CWL, a correlation-based sparsity allocator. Experiments report average Flores perplexity for 15 calibration languages at 50% sparsity for six models, downstream zero-shot tasks for Llama-8B, unseen-language perplexity for Llama-8B, sparsity sweeps, ablations, and an extension to RIA. The authors conclude that M-Wanda consistently improves multilingual retention at minimal additional cost and release code.

Significance. The paper targets a real and understudied problem: compression evaluation centered on English fails to capture multilingual degradation. The empirical scope is broad — six architectures, 15 calibration languages, 15 unseen languages, multiple sparsity levels, downstream tasks, ablations, and robustness checks — and the method is described precisely enough to reimplement. The released code and the systematic evaluation of typologically diverse languages are concrete strengths. If the reported gains survive a cleaned-up evaluation protocol, the paper would be a useful contribution to multilingual pruning and would support its call for multilingual benchmarks. The main reservation is that the headline comparison uses per-model hyperparameters selected without an explicit validation procedure, and the quantitative evidence for "consistently" is thinner for the smallest gains and for unseen languages outside Llama-8B.

major comments (3)
  1. [Appendix A / Table 1] Appendix A reports per-model "optimal" values of λ, ε, γ, and CWL block, obtained "after a small search", but it does not state the search objective or any use of a held-out set. Since Table 1 reports exactly the average Flores perplexity over the 15 calibration languages — the metric M-Wanda is designed to improve — selecting hyperparameters on this same metric can inflate the 2–7% improvements over Wanda. Wanda has no tuned hyperparameters, so the comparison is asymmetric. The authors should either fix the hyperparameters before seeing the test set, select them on a validation split, or report the full grid of results; without one of these, the claim in Section 5.2 that M-Wanda "consistently reduces perplexity" is not yet established.
  2. [Table 1 / Section 5.2.3] Table 1 reports a single average perplexity per model without error bars or repeated calibration sampling. The robustness experiment in Section 5.2.3 covers only Llama-8B (three seeds); for Aya-23-8B and Bloomz-7b1 the reported M-Wanda gain is only 2%, which is within the range one would expect from calibration-set variability. To support the "consistently" claim, please report standard deviations or interval estimates for all models in Table 1, or at least for the smallest improvements.
  3. [Section 5.2.1 / Figure 4] Section 5.2.1 and Figure 4 present unseen-language generalization results only for Llama-8B, but the text says "our method consistently reduces perplexity across all languages" and the abstract claims consistent improvement. Either report unseen-language results for the other five models, or restrict the generalization claim to Llama-8B and make that restriction explicit.
minor comments (5)
  1. [Figure 4] The caption should define the percentage decrease explicitly; as written, it is not clear whether the percentages are relative to Wanda or to the dense model.
  2. [Figure 5 / Section 5.2.1] The y-axis label in Figure 5 reads "Peplexity" and should be "Perplexity"; in Section 5.2.1, "Ukranian" should be "Ukrainian".
  3. [Appendix A / Table 5] The λ=6 entry for Bloomz-7b1 lies outside the displayed grid λ∈[0.02,0.2]; the separate grid for Bloomz should be stated in the table or caption, not only in the surrounding text.
  4. [Section 6] The sentence reporting that enhancing Wanda+OWL with variance and activation probability improves performance to 19.09 on Llama-8B refers to a configuration not shown in Table 3; please add it to the table or explicitly mark it as an additional ablation.
  5. [Section 7 / Equation 8] The notation for the row and column sums in Equation 8 is inconsistent with the prose; define P|W∗j| and P|Wi∗| precisely, for example as sums over columns and rows of W, respectively.

Circularity Check

1 steps flagged · score 6.0 of 10

Per-model hyperparameters in Appendix A are selected by a search whose results feed Table 1 directly, so the headline Flores perplexity gains are partly a selection artifact; XL-Sum, unseen-language, and downstream results reuse the same selected configurations and provide only partial independent support.

  1. fitted input called prediction [Appendix A (Table 5) and Table 1]
    "In Table 5, we report the optimal hyperparameters used when applying M-Wanda to each model. Note that these are the hyperparameters used for both the results on the Flores dataset, reported in Table 1, and the XL-Sum dataset, reported in Table 6. ... Optimal hyperparameters found for M-Wanda after a small search λ ∈ [0.02, 0.2], ϵ ∈ [5e-5, 1e-7 ], γ = [0.01, 0.04] and CWL block ∈ [attn, MLP] ."

    The headline result in Table 1 is the average Flores perplexity on the calibration languages, and Appendix A states that the per-model hyperparameters (λ, ϵ, γ, CWL block) were found by a 'small search' and are exactly the ones used for that Flores table. Because Wanda itself has no tuned free parameters, the comparison is asymmetric: each M-Wanda model is allowed to pick its best searched configuration on the very benchmark used for the reported improvement. The 2–7% Flores decreases therefore do not represent a fixed-configuration prediction; they are selected-configuration results.

full rationale

The paper's central derivation chain is not circular in the self-citation or uniqueness-theorem sense: M-Wanda is built directly on Wanda, OWL, and RIA, and the proposed CWL, variance, and activation-probability terms are self-contained modifications with explicit equations. The main circularity is the hyperparameter-selection loop. Appendix A reports per-model 'optimal' λ, ϵ, γ, and CWL block values found 'after a small search', and states that these are the hyperparameters used for the Flores results in Table 1, the paper's primary evidence for consistent perplexity improvement. No held-out validation split or nested selection is described, so the headline Flores gains conflate method quality with per-model tuning on the evaluation benchmark. The XL-Sum, unseen-language, and downstream-task results use the same tuned configurations and thus provide meaningful but partial independent support, since they were not themselves the search objective. Weighing the central Flores claim as partially selection-driven and the auxiliary results as partially independent, a circularity score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two modeling assumptions about neuron behavior (correlation marks cross-lingual sharing, and rare high-variance neurons are noise) and on four per-model hyperparameters tuned on the evaluation benchmark. No new entities (particles, forces, dimensions) are introduced.

free parameters (4)
  • lambda (variance scaling) = 0.02 to 0.2; 6 for Bloomz-7b1
    Balances the cross-lingual variance term against the activation norm in Eq. 6. Tuned per model via grid search on Flores perplexity (Appendix A).
  • epsilon (activation probability threshold) = 5e-5 or 1e-7; 0 for Bloomz-7b1
    Threshold for counting a neuron as active in Eq. 7. Tuned per model on the evaluation benchmark.
  • gamma (sparsity range) = 0.04 or 0.01
    Restricts layerwise sparsity ratios to [R-gamma, R+gamma] following OWL. Tuned per model.
  • CWL block (attention vs MLP) = attn for 5 models, MLP for Aya-23 and OLMo
    Discrete choice of which sublayer type is used for correlation-based sparsity allocation. Tuned per model.
assumptions (3)
  • domain assumption Layers with high inter-language activation correlation are more involved in cross-lingual sharing and should be pruned less.
    Stated explicitly as a hypothesis in Section 3.1; this drives the CWL sparsity allocation and is load-bearing for the method's effectiveness.
  • domain assumption High-variance neurons that rarely activate are noisy and should be filtered out.
    Motivates the activation-probability term in Eq. 7, Section 3.3. No direct evidence is provided that these neurons are actually noise.
  • domain assumption The measured input activations from 128 calibration samples (8 per non-English language) are representative of each language's true activation distribution.
    The method computes per-language means, variances, and activation probabilities from these samples; noisy estimates would undermine the pruning decisions. A 3-seed robustness check is provided for Llama-8B only.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M-Wanda: Improving One-Shot Pruning for Multilingual LLMs." pith.science (2026). https://pith.science/paper/BMAWQATV

@misc{pith2026250521171,
  author       = {Pith},
  title        = {Pith review of: M-Wanda: Improving One-Shot Pruning for Multilingual LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BMAWQATV}},
  note         = {Machine review of arXiv:2505.21171}
}
read the original abstract

Multilingual LLM performance is often critically dependent on model size. With an eye on efficiency, this has led to a surge in interest in one-shot pruning methods that retain the benefits of large-scale pretraining while shrinking the model size. However, as pruning tends to come with performance loss, it is important to understand the trade-offs between multilinguality and sparsification. In this work, we study multilingual performance under different sparsity constraints and show that moderate ratios already substantially harm performance. To help bridge this gap, we propose M-Wanda, a pruning method that models cross-lingual variation by incorporating language-aware activation statistics into its pruning criterion and dynamically adjusts layerwise sparsity based on cross-lingual importance. We show that M-Wanda consistently improves performance at minimal additional costs. We are the first to explicitly optimize pruning to retain multilingual performance, and hope to inspire future advances in multilingual pruning.

Figures

Figures reproduced from arXiv: 2505.21171 by the authors.

Figure 1
Figure 1. The effect of Wanda pruning under different sparsity ratios on the perplexity of each calibration language. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Performance in accuracy (%) given different sparsity ratios used on different sizes of Llama3. Zero-shot results are averaged across test languages per downstream task. 5 Results In Section 5.1, we first show how pruning under different sparsity constraints affects multilingual LLMs of different sizes. Motivated by these find￾ings, we show in Section 5.2 how our M-Wanda method can help mitigate some of the multiling… view at source ↗
Figure 3
Figure 3. Perplexity scores per language from Llama [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Relative percentage decrease in perplexity when using M-Wanda compared to Wanda for all 15 calibration [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Average perplexity scores across languages as [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Layerwise sparsity allocation using CWL. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The effect of higher sparsity ratio’s on the perplexity across languages. The calibration data is fully in [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Layerwise sparsity allocation by OWL. F Downstream task results per language 13 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 25 canonical work pages

  1. [1]

    Abhinav Bandari, Lu Yin, Cheng-Yu Hsieh, Ajay Jaiswal, Tianlong Chen, Li Shen, Ranjay Krishna, and Shiwei Liu. 2024. Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18089--18099

    work page 2024

  2. [2]

    Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. 2020. What is the state of neural network pruning? Proceedings of machine learning and systems, 2:129--146

    work page 2020

  3. [3]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross-lingual Sentence Representations . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475--2485

    work page 2018

  4. [4]

    John Dang, Shivalika Singh, Daniel D'souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, et al. 2024. Aya expanse: Combining research breakthroughs for a new multilingual frontier . arXiv preprint arXiv:2412.04261

    arXiv 2024

  5. [5]

    Jonathan Frankle and Michael Carbin. 2018. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks . In International Conference on Learning Representations

    work page 2018

  6. [6]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot . In International Conference on Machine Learning, pages 10323--10337. PMLR

    work page 2023

  7. [7]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  8. [8]

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Nai...

    work page 2024

Show all 36 references
  1. [9]

    Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28

    2015

  2. [10]

    Tahmid Hasan, Abhik Bhattacharjee, Md Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M Sohel Rahman, and Rifat Shahriyar. 2021. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages . In Findings of the Association for Computational Linguistic...

    2021

  3. [11]

    Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, and Xia Song. 2024. Scaling Laws for Multilingual Language Models . arXiv preprint arXiv:2410.12883

    2024 arXiv

  4. [12]

    Weizhong Huang, Yuxin Zhang, Xiawu Zheng, Fei Chao, and Rongrong Ji. 2025. Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective . arXiv preprint arXiv:2502.14770

    2025 arXiv

  5. [13]

    Yixin Ji, Yang Xiang, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. 2024. Beware of Calibration Data for Pruning Large Language Models . The Thirteenth International Conference on Learning Representations

    2024

  6. [14]

    Takeshi Kojima, Itsuki Okimura, Yusuke Iwasawa, Hitomi Yanaka, and Yutaka Matsuo. 2024. On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons . In Proceedings of the 2024 Conference of the North American Cha...

    2024

  7. [15]

    Simon Kurz, Jian-Jia Chen, Lucie Flek, and Zhixue Zhao. 2024. Investigating Language-Specific Calibration For Pruning Multilingual Large Language Models . arXiv preprint arXiv:2408.14398

    2024

  8. [16]

    Yann LeCun, John Denker, and Sara Solla. 1989. Optimal brain damage. Advances in neural information processing systems, 2

    1989

  9. [17]

    Lujun Li, Peijie Dong, Zhenheng Tang, Xiang Liu, Qiang Wang, Wenhan Luo, Wei Xue, Qifeng Liu, Xiaowen Chu, and Yike Guo. 2024. Discovering sparsity allocation for layer-wise pruning of large language models . Advances in Neural Information Processing Systems, 37:141292--141317

    2024

  10. [18]

    Chaitanya Malaviya, Graham Neubig, and Patrick Littell. 2017. Learning Language Representations for Typology Prediction . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2529--2535

    2017

  11. [19]

    Asit Mishra, Jorge Albericio Latorre, Jeff Pool, Darko Stosic, Dusan Stosic, Ganesh Venkatesh, Chong Yu, and Paulius Micikevicius. 2021. Accelerating sparse deep neural networks. arXiv preprint arXiv:2104.08378

    2021 arXiv

  12. [20]

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. 2023. Crosslingual Generalization through Multitask Finetuning . In The 61st Annual Meeting Of The Association ...

    2023

  13. [21]

    Kelechi Ogueji, Orevaoghene Ahia, Gbemileke Onilude, Sebastian Gehrmann, Sara Hooker, and Julia Kreutzer. 2022. Intriguing Properties of Compression on Multilingual Models . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9092--9110

    2022

  14. [22]

    Denis Paperno, Germ \'a n Kruszewski, Angeliki Lazaridou, Ngoc-Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fern \'a ndez. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context . In Proceedings of the 54th Annu...

    2016

  15. [23]

    Edoardo Maria Ponti, Goran Glava s , Olga Majewska, Qianchu Liu, Ivan Vuli \'c , and Anna Korhonen. 2020. XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2...

    2020

  16. [24]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of machine learning research, 21(140):1--67

    2020

  17. [25]

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. The Twelfth International Conference on Learning Representations

    2023

  18. [26]

    Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Wayne Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models . In Proceedings of the 62nd Annual Meeting of the Association for C...

    2024

  19. [27]

    Miles Williams and Nikolaos Aletras. 2024. On the Impact of Calibration Data in Post-training Quantization and Pruning . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10100--10118

    2024

  20. [28]

    Ge Yang, Changyi He, Jinyang Guo, Jianyu Wu, Yifu Ding, Aishan Liu, Haotong Qin, Pengliang Ji, and Xianglong Liu. 2024. LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment . In The Thirty-eight Conference on Neural Information Processing Systems D...

    2024

  21. [29]

    Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Na...

    2019

  22. [30]

    Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Kumar Jaiswal, Mykola Pechenizkiy, Yi Liang, et al. 2024. Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity . In International Conference on Mac...

    2024

  23. [31]

    Hongchuan Zeng, Hongshen Xu, Lu Chen, and Kai Yu. 2024. Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (L...

    2024

  24. [32]

    Yingtao Zhang, Haoli Bai, Haokun Lin, Jialin Zhao, Lu Hou, and Carlo Vittorio Cannistraci. 2024. Plug-and-play: An efficient post-training pruning method for large language models . In The Twelfth International Conference on Learning Representations

    2024

  25. [33]

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. How do Large Language Models Handle Multilingualism? In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  26. [34]

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2024. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12:1556--1577

    2024

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.