Pith. sign in

REVIEW 2 major objections 6 minor 41 references

QPruner: Probabilistic Decision Quantization for Structured Pruning in Large Language Models

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read QPruner weds structured pruning to per-layer 4/8-bit quantization, and at 50% pruning on LLaMA-7B it reports higher accuracy than the half-precision baseline on BoolQ while using about 30% less memory.

desk verdict Sensible engineering integration with a fatal evaluation flaw: QPruner3's accuracy gains are likely test-set-selected maxima, so the central claim is unsupported. read the letter →

arxiv 2412.11629 v1 pith:UVEG55YV submitted 2024-12-16 cs.LG

Changhai Zhou , Yuhua Zhou , Shijie Han , Qian Qiao , Hongguang Li This is my paper · ORCID
classification cs.LG
keywords structuredpruningmixed-precisionquantizationBayesianoptimizationlargelanguagemodelsparameter-efficientfine-tuninglow-rankadaptationmutualinformationmemoryefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Structured pruning removes whole groups of neurons from an LLM, but the accuracy loss it causes normally has to be repaired by fine-tuning, which costs more memory than pruning saved. QPruner treats pruning, quantization, and fine-tuning as one pipeline: after pruning, each layer is assigned a 4- or 8-bit precision according to how strongly its output predicts the task answer, and a Bayesian optimizer then searches the layer-wise precision assignments under a memory ceiling. The paper's central result is that this combined scheme can beat the standard pruned half-precision baseline on several zero-shot commonsense benchmarks while cutting peak memory by roughly 30% at aggressive pruning rates. Concretely, on LLaMA-7B pruned to 50%, the optimized QPruner3 configuration reports 48.37% on BoolQ versus 43.76% for the baseline, using 16.65 GB instead of 23.89 GB. If the result holds, resource-constrained users can fine-tune and deploy smaller, quantized LLMs without the accuracy penalty usually associated with structured pruning.

What carries the argument

The load-bearing object is the bit-width vector $b=[B_1,\ldots,B_L]$ with each $B_i$ restricted to 4 or 8 bits and constrained by a total memory budget $M(b)\le M_{\max}$. The allocation runs in two stages: mutual information $I(X_i;Y)$ ranks every layer's contribution to the task and fixes an initial precision assignment, and a Gaussian-process model of the map from bit-width vectors to (accuracy, memory) then proposes new configurations through an acquisition function $\alpha(b)$, fine-tunes each candidate, and iterates toward a Pareto front. Performance recovery uses low-rank adapters initialized by alternating quantization and low-rank approximation, so only small adapter matrices are trained.

What would settle it

Run the full QPruner pipeline on LLaMA-7B at 50% pruning with the seven downstream tasks held out during Bayesian optimization, select the bit-width configuration on a separate validation set, and report the held-out test accuracy; if the configuration chosen without test information no longer beats LLM-Pruner on BoolQ or saves less than 30% memory, the central claim as stated fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that after structured pruning, the model's layers are no longer equally important, so a single uniform bit width either wastes memory or damages prediction. QPruner discovers that per-layer precision should be driven by the mutual information $I(X;Y)$ between each layer's output and the model's final prediction, then refined by Bayesian optimization that trades accuracy against memory. With this allocation, the quantized pruned model can be recovered with low-rank adapter fine-tuning and in several configurations ends up more accurate than the unquantized pruned baseline at lower peak memory. The authors state the advantage grows with pruning rate: at 50% pruning on LLaMA-7B the best configuration improves BoolQ accuracy from 43.76% to 48.37% and lowers memory from 23.89 GB to 16.65 GB, about a 30% reduction.

Load-bearing premise

The Bayesian optimizer's performance scores appear to come from the same benchmark tasks used to report final accuracy, with no described validation split, so the reported QPruner3 numbers may be selected maxima rather than unbiased estimates.

Editorial extensions

If this is right

  • At 50% pruning on LLaMA-7B, the optimized QPruner3 configuration reports 48.37% on BoolQ versus 43.76% for the LLM-Pruner baseline, with peak memory dropping from 23.89 GB to 16.65 GB, about a 30% reduction.
  • On Vicuna-7B at 50% pruning, QPruner3 also beats the baseline on BoolQ (61.56% vs 59.51%) and WinoGrande (53.39% vs 52.01%) while using less than two-thirds of the baseline's memory.
  • The ablation results show the main gain comes from the mixed-precision allocation itself, since replacing NF4 with FP4 or changing the low-rank initialization changes scores only slightly.
  • Bayesian refinement (QPruner3 over QPruner2) usually improves or preserves accuracy, with the largest gaps appearing at 30% and 50% pruning where layer imbalances are strongest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same per-layer importance signal could be reused to choose pruning ratios per layer, not just quantization widths, making the two compression decisions jointly optimal.
  • A decisive check is to hold out all seven benchmarks during Bayesian optimization: if the 'best' configuration is selected on a validation set instead, the reported gains may shrink or vanish.
  • Because only 4-bit and 8-bit options are considered, the Pareto front may miss useful 3-bit or mixed 4/8/16 configurations for extremely memory-starved deployments.
  • The mutual-information importance estimator is cheap and could replace Hessian-based sensitivity in other structured-pruning pipelines that currently fine-tune at full precision.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes QPruner, a framework that combines structured pruning (using LLM-Pruner as the pruning backbone), layer-wise mixed-precision quantization initialized by mutual information between layer outputs and model predictions, and Bayesian optimization to refine the per-layer bit-width configuration, followed by LoRA/LoftQ fine-tuning. Experiments on LLaMA-7B, LLaMA-13B, and Vicuna-7B at 20%, 30%, and 50% pruning rates report peak memory savings of roughly 30% or more and, for the Bayesian-optimized variant QPruner3, accuracy improvements over LLM-Pruner on several zero-shot commonsense benchmarks. The paper claims that QPruner can surpass baseline methods in accuracy by up to 6% while saving at least 30% of memory.

Significance. If the accuracy gains were genuine, QPruner would be a practically valuable recipe for low-memory fine-tuning of pruned LLMs. The memory-savings component (Table 1) is consistent and is a real contribution: quantizing to 4/8 bits with LoftQ initialization reduces peak memory substantially. The accuracy claims, however, are not credible as reported because the Bayesian optimizer selects configurations on the same test sets used for final evaluation (Appendix D, Figures 3-4). Thus QPruner3's numbers are selected maxima, and the comparison to LLM-Pruner is unfair. The mutual-information-only and uniform variants (QPruner1/2) are less affected and show modest gains, but the headline 6% improvement is unsupported.

major comments (2)
  1. [Section 3.2, Algorithm 1; Appendix D; Figures 3-4; Table 1] The Bayesian optimization objective P(b) is the model's accuracy after fine-tuning on a given configuration, and Appendix D reveals that the Pareto-front data points are generated per benchmark (BoolQ, WinoGrande, ARC-c, ARC-e, HellaSwag, OBQA, PIQA). These are exactly the seven tasks in Table 1, and no validation split or held-out test set is mentioned anywhere in Section 4 or the appendices. Consequently, each QPruner3 entry in Table 1 is the maximum over the 50 evaluated configurations on that task's test set. Comparing those selected maxima to LLM-Pruner, which is not subjected to per-task selection, does not measure generalization; the 48.37 vs. 43.76 BoolQ difference at 50% pruning is exactly the kind of improvement one would expect from test-set selection. This is load-bearing for the central accuracy claim and invalidates the QPruner3 rows as unbiased evidence.
  2. [Section 4.1 and Appendix D] QPruner3 is described as 'the mixed-precision quantization after further optimization using Bayesian methods based on QPruner2,' and Algorithm 1 initializes b0 from the mutual-information computation. Appendix D, however, states that for the reported LLaMA-7B 50%-pruning experiments, the GP was initialized with '10 sets of configurations' whose precisions were 'randomly selected between 4-bit and 8-bit,' with no indication that the QPruner2 configuration was included. The reported QPruner3 results therefore do not actually demonstrate the Bayesian refinement of the mutual-information allocation that the method section promises; they demonstrate optimization from a random starting population. This inconsistency should be resolved, and the incremental benefit of the mutual-information initialization should be measured.
minor comments (6)
  1. [Overall structure] The paper contains two separate 'Limitation' sections, one directly before the references and one as Appendix G; they should be merged or cross-referenced.
  2. [Table 3] Table 3 reports memory values as parenthetical numbers (e.g., '61.93(41.32)') without a header specifying that the parenthetical is peak memory in GB; this makes the table difficult to read, and the memory value 41.32 GB for LLM-Pruner is larger than the unquantized model's Table 1 memory, which should be explained.
  3. [Section 3.2, Eq. (7)] Equation (7) computes mutual information from layer outputs, but the required discretization of continuous activations (number of bins or a density estimator) is not specified; this hampers reproducibility.
  4. [Algorithm 1] Algorithm 1 refers to an acquisition function α(b) but does not state which acquisition function is used (e.g., expected improvement, UCB); the Optuna settings in Appendix B are also not detailed enough to reproduce the GP model.
  5. [Section 4.1 and Abstract] Section 4 states 'QPruner performes' (typo for 'performs') and the abstract's claim of 'up to 6%' accuracy improvement is never precisely anchored to a specific baseline-task-pruning-rate triple in the main text.
  6. [Appendix F] Appendix F gives usage instructions but no public code repository link; providing the implementation would be necessary for the claimed results to be auditable.

Circularity Check

1 steps flagged · score 6.0 of 10

QPruner3's reported accuracy is the optimized objective P(b): the Bayesian search measures performance on the same benchmarks later reported in Table 1, with no validation split described.

  1. fitted input called prediction [Section 3.2 Algorithm 1; Section 4.1; Appendix D; Figure 3]
    "QPruner3 refers to the mixed-precision quantization after further optimization using Bayesian methods based on QPruner2. ... Apply bt+1 to pruned model and fine-tune. Measure P (bt+1), M (bt+1) ... We set the total number of iterations for QPruner to 40 (resulting in 50 data points for constructing the Pareto front) to ensure the best configuration was found. ... Figure 3: Pareto-front scatter plots for BoolQ and WinoGrande with 50 data points."

    Algorithm 1 defines the optimization objective as P(b), the model performance measured after each candidate bit-width configuration is fine-tuned, and Section 4.1 identifies QPruner3 as the result of 'further optimization using Bayesian methods.' Appendix D says the search generates 50 data points 'to ensure the best configuration was found,' and Figure 3 plots those Pareto-front points for BoolQ and WinoGrande, the same benchmarks whose final accuracies appear in Table 1. No validation split is described anywhere between the data used to measure P(b) and the data used to report final accuracy.

full rationale

The circular component is confined to QPruner3's accuracy claims, which are selected maxima from the Bayesian optimization loop's own objective P(b), measured on the same benchmark tasks that Table 1 reports. The paper never states that P(b) is evaluated on a separate validation or calibration split; instead, Appendix D and Figure 3 tie the Pareto-front search directly to BoolQ and WinoGrande, and Table 1 reports the resulting accuracies. This makes the reported QPruner3 improvement a fitted maximum rather than an independent prediction. QPruner1 and QPruner2, which use fixed uniform and mutual-information rules, are not affected by this circularity, and the memory-savings claims are independent of the accuracy objective. There is no load-bearing self-citation in the paper. The score is 6 because the central accuracy claim reduces by construction to the optimized objective, but the framework's memory benefits and non-Bayesian variants remain independently meaningful.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework rests on search over bit-width configurations; the main fitted object is the per-layer bit-width vector selected by BO. The paper does not provide code, seeds, or validation splits, so the ledger is dominated by hand-chosen constraints and unvalidated importance proxies.

free parameters (4)
  • Layer-wise bit-width configuration b per benchmark = not disclosed; selected from 50 Pareto-front points
    The central table reports QPruner3 after Bayesian optimization. The optimizer chooses the 4/8-bit assignment that maximizes P(b) on each benchmark, so the reported accuracy is a fitted value, not an independent estimate.
  • Average bit-width or memory constraint Mmax = implicit from memory budget; 8-bit layers kept below 25%
    Hand-chosen constraint in the Implementation Details section; it shapes the feasible set for the search and is not derived from data.
  • Bayesian optimization budget and GP initialization count = 40 iterations, 10 initial configs, 50 total points
    Appendix D fixes these by hand; results depend on search effort, and no sensitivity analysis is given.
  • Mutual information discretization parameters = not specified
    Section 3.2 Eq. 7 requires p(x,y) for continuous activations; no binning or sample count is given, so MI values are effectively unspecified free choices.
assumptions (4)
  • standard math Second-order Taylor expansion of the loss approximates group importance for pruning (Eq. 5-6).
    Inherited from LLM-Pruner; assumes loss is locally quadratic and gradient/Hessian estimates are accurate.
  • domain assumption Mutual information between a layer's output and the final prediction measures that layer's importance for quantization.
    Section 3.2 introduces I(X;Y) as importance but gives no validation that MI correlates with sensitivity to bit-width reduction.
  • ad hoc to paper A bit-width configuration selected by Bayesian optimization on P(b) gives an unbiased estimate of final benchmark performance.
    The paper never describes a train/validation/test split; Appendix D suggests P(b) is measured on the benchmark itself, so this assumption is load-bearing and unsupported.
  • domain assumption Quantized models fine-tuned with LoftQ recover performance similarly across layers.
    The performance recovery section assumes LoftQ initialization repairs quantization error after pruning; only one iteration is used without analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QPruner: Probabilistic Decision Quantization for Structured Pruning in Large Language Models." pith.science (2026). https://pith.science/paper/UVEG55YV

@misc{pith2026241211629,
  author       = {Pith},
  title        = {Pith review of: QPruner: Probabilistic Decision Quantization for Structured Pruning in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVEG55YV}},
  note         = {Machine review of arXiv:2412.11629}
}
read the original abstract

The rise of large language models (LLMs) has significantly advanced various natural language processing (NLP) tasks. However, the resource demands of these models pose substantial challenges. Structured pruning is an effective approach to reducing model size, but it often results in significant accuracy degradation, necessitating parameter updates to adapt. Unfortunately, such fine-tuning requires substantial memory, which limits its applicability. To address these challenges, we introduce quantization into the structured pruning framework to reduce memory consumption during both fine-tuning and inference. However, the combined errors from pruning and quantization increase the difficulty of fine-tuning, requiring a more refined quantization scheme. To this end, we propose QPruner, a novel framework that employs structured pruning to reduce model size, followed by a layer-wise mixed-precision quantization scheme. Quantization precisions are assigned to each layer based on their importance to the target task, and Bayesian optimization is employed to refine precision allocation strategies, ensuring a balance between model accuracy and memory efficiency. Extensive experiments on benchmark datasets demonstrate that QPruner significantly outperforms existing methods in memory savings while maintaining or improving model performance.

Figures

Figures reproduced from arXiv: 2412.11629 by the authors.

Figure 1
Figure 1. Comparison of accuracy and memory usage across different fine-tuning configurations for multiple tasks. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the QPruner framework. A popular structured pruning method is LLM￾Pruner (Ma et al., 2023), which first identifies dependencies between neurons and groups them, then removes weights based on their importance. Let Ni and Nj be two neurons in the model. If Nj ∈ Out(Ni) and Deg−(Nj ) = 1, then Nj is dependent on Ni . Similarly, if Ni ∈ In(Nj ) and Deg+(Ni) = 1, then Ni is dependent on Nj . Based on this pri… view at source ↗
Figure 3
Figure 3. Pareto-front scatter plots for BoolQ and Wino [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Pareto-front scatter plots for different Downstream Tasks [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 13 canonical work pages

  1. [1]

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865--10873

    2024

  2. [2]

    Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. 2020. Binarybert: Pushing the limit of bert quantization. arXiv preprint arXiv:2012.15701

    arXiv 2020

  3. [3]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  4. [4]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pa...

    2019

  5. [5]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    arXiv 2018

  6. [6]

    Xiang Deng, Vasilisa Bashlovkina, Feng Han, Simon Baumgartner, and Michael Bendersky. 2023. What do llms know about financial markets? a case study on reddit market sentiment analysis. In Companion Proceedings of the ACM Web Conference 2023, pages 107--110

    work page 2023

  7. [7]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314

    arXiv 2023

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

    2024

Show all 41 references
  1. [9]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323--10337. PMLR

    2023

  2. [10]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    2022 arXiv

  3. [11]

    Peter I Frazier. 2018. Bayesian optimization. In Recent advances in optimization and modeling of contemporary problems, pages 255--278. Informs

    2018

  4. [12]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2023 doi

  5. [13]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2023. Minillm: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations

    2023

  6. [14]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  7. [15]

    Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natura...

    2023

  8. [16]

    Janghwan Lee, Minsoo Kim, Seungcheol Baek, Seok Hwang, Wonyong Sung, and Jungwook Choi. 2023. Enhancing computation efficiency in large language models through weight and activation quantization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pr...

    2023

  9. [17]

    Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2023. Loftq: Lora-fine-tuning-aware quantization for large language models. In The Twelfth International Conference on Learning Representations

    2023

  10. [18]

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023. Llm-qat: Data-free quantization aware training for large language models. arXiv preprint arXiv:2305.17888

    2023 arXiv

  11. [19]

    Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270

    2018 arXiv

  12. [20]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702--21720

    2023

  13. [21]

    Meng. 2024. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948

    2024 arXiv

  14. [22]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381--2391

    2018

  15. [23]

    Zeping Min and Jinbo Wang. 2023. Exploring the integration of large language models into automatic speech recognition systems: An empirical study. In International Conference on Neural Information Processing, pages 69--84. Springer

    2023

  16. [24]

    Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz. 2019. Importance estimation for neural network pruning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11264--11272

    2019

  17. [25]

    Dheeraj Peri, Jhalak Patel, and Josh Park. 2020. Deploying quantization-aware trained networks using tensorrt. In GPU Technology Conference

    2020

  18. [26]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  19. [27]

    Michael Santacroce, Zixin Wen, Yelong Shen, and Yuanzhi Li. 2023. What matters in the structured pruning of generative language models? arXiv preprint arXiv:2302.03773

    2023 arXiv

  20. [28]

    Shoetsu Sato, Jin Sakuma, Naoki Yoshinaga, Masashi Toyoda, and Masaru Kitsuregawa. 2020. Vocabulary adaptation for domain adaptation in neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4269--4279

    2020

  21. [29]

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2023. Omniquant: Omnidirectionally calibrated quantization for large language models. In The Twelfth International Conference on Learning Representations

    2023

  22. [30]

    Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2020. Q-bert: Hessian based ultra low precision quantization of bert. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8815--8821

    2020

  23. [31]

    Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, and Jie Tang. 2023. Gkd: A general knowledge distillation framework for large-scale pre-trained language model. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguist...

    2023

  24. [32]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  25. [33]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  26. [34]

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. 2023. Sheared llama: Accelerating language model pre-training via structured pruning. In The Twelfth International Conference on Learning Representations

    2023

  27. [35]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  28. [36]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4800

    2019

  29. [37]

    Biao Zhang, Barry Haddow, and Alexandra Birch. 2023 a . Prompting large language model for machine translation: A case study. In International Conference on Machine Learning, pages 41092--41110. PMLR

    2023

  30. [38]

    Boyu Zhang, Hongyang Yang, Tianyu Zhou, Muhammad Ali Babar, and Xiao-Yang Liu. 2023 b . Enhancing financial sentiment analysis via retrieval augmented large language models. In Proceedings of the Fourth ACM International Conference on AI in Finance, pages 349--356

    2023

  31. [39]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36

    2024

  32. [40]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.