Pith. sign in

REVIEW 69 references

Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis

T0 review · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Quaff shows that activation outlier channels keep their spatial positions during LLM fine-tuning, and exploits this stability to cut fine-tuning memory and latency with INT8 quantization while matching or beating full-precision accuracy.

arxiv 2505.14742 v2 pith:HZWGXXPZ submitted 2025-05-20 cs.LG cs.AI

Hong Huang , Dapeng Wu This is my paper
classification cs.LGcs.AI
keywords fine-tuningquaffacrossactivationmemoryosshoutlierquantized
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are usually fine-tuned with floating-point weights, which takes memory and time. One way to make fine-tuning lighter is to store weights and activations as 8-bit integers, but this fails when some activation channels have huge outlier values that dominate the quantization scale. Prior methods either use static scaling factors that become stale as the model changes, or dynamic scaling that requires keeping full-precision weights around. Quaff's idea is to first identify the channels that contain outliers using a small calibration set, and then assume these channels stay in the same positions during training. For those stable outlier channels only, Quaff computes a scaling factor each step, smoothed by a momentum term, and keeps the corresponding slice of the weight matrix in full precision. The rest of the weights are quantized to INT8 and never need to be dequantized. This decouples weight quantization from activation quantization and avoids global rescaling. The paper validates the stability hypothesis by measuring how often real-time detected outliers fall inside the pre-identified channels, reporting above 90% hit rates. Experiments on ten benchmarks with three model families and four PEFT methods show Quaff with INT8 quantization roughly matches or slightly exceeds full-precision accuracy while reducing memory and latency. The main comparison is against other weight-activation quantization schemes, not against QLoRA-style weight-only quantization, which is the most common practical baseline.
Extended reading notes

Core claim

The load-bearing claim is the Outlier Spatial Stability Hypothesis: 'During fine-tuning, certain activation outlier channels retain stable spatial positions across training iterations' (Sec 3.2). The paper argues that given this stability, Eq 5's correction term can be computed from a small full-precision submatrix W_O, enabling targeted momentum scaling, INT8 weights and activations, no global rescaling, and less than 5% overhead.

Load-bearing premise

The most fragile premise is that the outlier channel set O identified once on 512 OIG/Chip2 samples, using a 100x-mean threshold and layer budgets of 0.03%/4%/10%, remains valid for other datasets and that the scale factor beta_i = max(1, sqrt(max|X_i| / max|W_i|)) with gamma=0.2 suffices through training. The hit-rate validation of OSSH depends on an unstated real-time outlier detection rule, so the reported >90% stability is not fully defined and may be partly guaranteed by the large per-layer budgets.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on a small set of hand-chosen hyperparameters (threshold, budgets, momentum, calibration set) and on the empirical validity of the OSSH. No new physical or mathematical entities are introduced.

free parameters (4)
  • outlier threshold = 100x mean activation
    Used in Eq 6 to decide whether a channel is an outlier; no sensitivity analysis is provided for this threshold.
  • layer-wise outlier budget = q/k/v/up_proj 0.03%, o_proj 4%, down_proj 10%; overall <5%
    Chosen empirically to keep total overhead below 5% and hit rates above 90%; uniform allocation drops hit rates below 50% (Fig 8).
  • momentum coefficient gamma = 0.2
    Set in Sec E for Eq 7 without a sweep or sensitivity study; the momentum ablation in Table 4 only compares with and without momentum.
  • calibration set size = 512 samples from OIG/Chip2
    Used for outlier identification; cross-dataset results in Table 3 are limited to three datasets.
assumptions (4)
  • domain assumption Quantization error in LLMs is dominated by a small set of channel-wise activation outliers.
    Standard result from prior work (Dettmers et al., 2022; Xiao et al., 2023) used to justify channel-wise scaling in Eq 3.
  • domain assumption A per-channel scalar scale of the form sqrt(max|X|/max|W|), as in SmoothQuant, sufficiently suppresses outliers when applied to the outlier channels.
    Adopted in Eq 8 without derivation; the paper inherits this from SmoothQuant.
  • domain assumption The pre-identified outlier channel set O remains valid during fine-tuning (OSSH).
    Central hypothesis of the paper; validated empirically via hit rates, but not proven; the dynamic detection rule used for validation is unstated.
  • standard math Symmetric round-to-nearest uniform quantization (Eq 1) is an adequate model of the INT8 arithmetic used in practice.
    Common assumption in quantization papers; the actual bitsandbytes kernel behavior may differ.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis." pith.science (2026). https://pith.science/paper/HZWGXXPZ

@misc{pith2026250514742,
  author       = {Pith},
  title        = {Pith review of: Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HZWGXXPZ}},
  note         = {Machine review of arXiv:2505.14742}
}
read the original abstract

Large language models (LLMs) have made exciting achievements across various domains, yet their deployment on resource-constrained personal devices remains hindered by the prohibitive computational and memory demands of task-specific fine-tuning. While quantization offers a pathway to efficiency, existing methods struggle to balance performance and overhead, either incurring high computational/memory costs or failing to address activation outliers, a critical bottleneck in quantized fine-tuning. To address these challenges, we propose the Outlier Spatial Stability Hypothesis (OSSH): During fine-tuning, certain activation outlier channels retain stable spatial positions across training iterations. Building on OSSH, we propose Quaff, a Quantized parameter-efficient fine-tuning framework for LLMs, optimizing low-precision activation representations through targeted momentum scaling. Quaff dynamically suppresses outliers exclusively in invariant channels using lightweight operations, eliminating full-precision weight storage and global rescaling while reducing quantization errors. Extensive experiments across ten benchmarks validate OSSH and demonstrate Quaff's efficacy. Specifically, on the GPQA reasoning benchmark, Quaff achieves a 1.73x latency reduction and 30% memory savings over full-precision fine-tuning while improving accuracy by 0.6% on the Phi-3 model, reconciling the triple trade-off between efficiency, performance, and deployability. By enabling consumer-grade GPU fine-tuning (e.g., RTX 2080 Super) without sacrificing model utility, Quaff democratizes personalized LLM deployment. The code is available at https://github.com/Little0o0/Quaff.git.

Figures

Figures reproduced from arXiv: 2505.14742 by the authors.

Figure 1
Figure 1. Comparison of accuracy, average latency per [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Spatial stability of outlier channels: activation magnitude distribution during fine-tuning, demonstrating [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The average hit rate of real-time versus prede [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of accuracy, latency, and memory footprints between our proposed Quaff and various WAQ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Accuracy and fine-tuning costs on the GPQA dataset using Phi3-3.8B with different fine-tuning strategies. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Results of LoRA fine-tuning on LAMBADA dataset with input/output size of 4K on different models. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Average hit rate of real-time vs. predefined [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Average hit rate of real-time vs. uniformly distributed predefined outlier channel indices across layers in Phi3-3.8B during fine-tuning on OIG/Chip2. core linear layers in transformer (Waswani et al., 2017; Devlin et al., 2018) of LLMs: 1. Atten￾tion projections: q_pr…
Figure 10
Figure 10. Figure 10: Pearson similarity between static and dynamic scaling factors (top 1%) across layers in LLaMA2-7B during fine-tuning on OIG/Chip2. D More Experiment Analysis D.1 The Impact of Long Context To explore The impact of Long Context, we in￾vestigated outlier stability on Ph…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 17 canonical work pages

  1. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  2. [2]

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319

    arXiv 2019

  3. [3]

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. Quarot: Outlier-free 4-bit inference in rotated llms. arXiv preprint arXiv:2404.00456

    arXiv 2024

  4. [4]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  5. [5]

    Gaurang Bharti. 2023. gbharti/finance-alpaca. https://huggingface.co/datasets/gbharti/finance-alpaca

    work page 2023

  6. [6]

    Keyan Cao, Yefan Liu, Gongjie Meng, and Qimeng Sun. 2020. An overview on edge computing research. IEEE access, 8:85714--85728

    work page 2020

  7. [7]

    Ning Chen, Tie Qiu, Xiaobo Zhou, Songwei Zhang, Weisheng Si, and Dapeng Oliver Wu. 2024. A distributed co-evolutionary optimization method with motif for large-scale iot robustness. IEEE/ACM Transactions on Networking

    work page 2024

  8. [8]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Llm. int8 (): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339

    arXiv 2022

Show all 69 references
  1. [9]

    Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2021. 8-bit optimizers via block-wise quantization. arXiv preprint arXiv:2110.02861

    2021 arXiv

  2. [10]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

    2024

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  4. [12]

    Luciano Floridi and Massimo Chiriatti. 2020. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681--694

    2020

  5. [13]

    Forman and John Zahorjan

    George H. Forman and John Zahorjan. 1994. The challenges of mobile computing. Computer, 27(4):38--47

    1994

  6. [14]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    2022 arXiv

  7. [15]

    Zihao Fu, Anthony Man-Cho So, and Nigel Collier. 2023. A stability analysis of fine-tuning a pre-trained model. arXiv preprint arXiv:2301.09820

    2023 arXiv

  8. [16]

    Han Guo, Philip Greengard, Eric P Xing, and Yoon Kim. 2023. Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning. arXiv preprint arXiv:2311.12023

    2023 arXiv

  9. [17]

    Yefei He, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. 2023. Efficientdm: Efficient quantization-aware fine-tuning of low-bit diffusion models. arXiv preprint arXiv:2310.03270

    2023 arXiv

  10. [18]

    Jung Hwan Heo, Jeonghoon Kim, Beomseok Kwon, Byeongwook Kim, Se Jung Kwon, and Dongsoo Lee. 2023. Rethinking channel dimensions to isolate outliers for low-bit weight quantization of large language models. arXiv preprint arXiv:2309.15531

    2023 arXiv

  11. [19]

    John Hewitt and Christopher D Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Sho...

    2019

  12. [20]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790--2799. PMLR

    2019

  13. [21]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  14. [22]

    Hong Huang, Hai Yang, Yuan Chen, Jiaxun Ye, and Dapeng Wu. 2025. Fedrts: Federated robust pruning via combinatorial thompson sampling. arXiv preprint arXiv:2501.19122

    2025

  15. [23]

    Hong Huang, Lan Zhang, Chaoyue Sun, Ruogu Fang, Xiaoyong Yuan, and Dapeng Wu. 2023. Distributed pruning towards tiny neural networks in federated learning. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS), pages 190--201. IEEE

    2023

  16. [24]

    Hong Huang, Weiming Zhuang, Chen Chen, and Lingjuan Lyu. 2024 a . Fedmef: Towards memory-efficient federated dynamic pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27548--27557

    2024

  17. [25]

    Xijie Huang, Zechun Liu, Shih-Yang Liu, and Kwang-Ting Cheng. 2024 b . Rolora: Fine-tuning rotated outlier-free llms for effective weight-activation quantization. arXiv preprint arXiv:2407.08044

    2024 arXiv

  18. [26]

    Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2018. Quantized neural networks: Training neural networks with low precision weights and activations. Journal of Machine Learning Research, 18(187):1--30

    2018

  19. [27]

    Tomasz Imielinski and Henry F Korth. 1996. Mobile computing, volume 353. Springer Science & Business Media

    1996

  20. [28]

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vis...

    2018

  21. [29]

    Aojie Jiang, Li Du, and Yuan Du. 2024. Groupq: Group-wise quantization with multi-objective optimization for cnn accelerators. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

    2024

  22. [30]

    Joseph Kampeas, Yury Nahshan, Hanoch Kremer, Gil Lederman, Shira Zaloshinski, Zheng Li, and Emir Haleva. 2023. Rotation invariant quantization for model compression. arXiv preprint arXiv:2303.03106

    2023 arXiv

  23. [31]

    Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. 2024. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization. Advances in Neural Information Processing Systems, 36

    2024

  24. [32]

    Jiwoo Kim, Joonhyung Lee, Gunho Park, Byeongwook Kim, Se Jung Kwon, Dongsoo Lee, and Youngjoo Lee. 2025. An investigation of fp8 across accelerators for llm inference. arXiv preprint arXiv:2502.01070

    2025 arXiv

  25. [33]

    o ksal, Timo Schick, Anna Korhonen, and Hinrich Sch \

    Abdullatif K \"o ksal, Timo Schick, Anna Korhonen, and Hinrich Sch \"u tze. 2023. Longform: Optimizing instruction tuning for long text generation with corpus extraction. arXiv preprint arXiv:2304.08460

    2023 arXiv

  26. [34]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neura...

    2024

  27. [35]

    Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, and Dongsoo Lee. 2022. Alphatuning: Quantization-aware parameter-efficient adaptation of large-scale pre-trained language models. arXiv preprint arXiv:2210.03858

    2022 arXiv

  28. [36]

    LAION. 2023. Open-instruction-generalist dataset. https://github.com/LAION-AI/Open-Instruction-Generalist

    2023

  29. [37]

    Changhun Lee, Jun-gyu Jin, Younghyun Cho, and Eunhyeok Park. 2024 a . Qeft: Quantization for efficient fine-tuning of llms. arXiv preprint arXiv:2410.08661

    2024 arXiv

  30. [38]

    Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. 2024 b . Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13355--13364

    2024

  31. [39]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

    2021 arXiv

  32. [40]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

    2021 arXiv

  33. [41]

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. 2023. Loftq: Lora-fine-tuning-aware quantization for large language models. arXiv preprint arXiv:2310.08659

    2023 arXiv

  34. [42]

    Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. 2025. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. Advances in Neural Information Processing Systems, 37:87766--87800

    2025

  35. [43]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. 2023. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978

    2023 arXiv

  36. [44]

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950--1965

    2022

  37. [45]

    Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. 2023. Qllm: Accurate and efficient low-bitwidth quantization for large language models. arXiv preprint arXiv:2310.08041

    2023 arXiv

  38. [46]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602

    2021 arXiv

  39. [47]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024. Spinquant--llm quantization with learned rotations. arXiv preprint arXiv:2405.16406

    2024 arXiv

  40. [48]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273--1282. PMLR

    2017

  41. [49]

    Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. 2020. On the stability of fine-tuning bert: Misconceptions, explanations, and strong baselines. arXiv preprint arXiv:2006.04884

    2020 arXiv

  42. [50]

    Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. 2016. https://doi.org/10.5281/zenodo.2630551 The lambada dataset

    2016 doi

  43. [51]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022

    2023 arXiv

  44. [52]

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2023. Omniquant: Omnidirectionally calibrated quantization for large language models. arXiv preprint arXiv:2308.13137

    2023 arXiv

  45. [53]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  46. [54]

    Jinguang Wang, Yuexi Yin, Haifeng Sun, Qi Qi, Jingyu Wang, Zirui Zhuang, Tingting Yang, and Jianxin Liao. 2024 a . Outliertune: Efficient channel-wise quantization for large language models. arXiv preprint arXiv:2406.18832

    2024 arXiv

  47. [55]

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560

    2022 arXiv

  48. [56]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024 b . Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574

    2024 arXiv

  49. [57]

    A Waswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A Gomez, L Kaiser, and I Polosukhin. 2017. Attention is all you need. In NIPS

    2017

  50. [58]

    Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, and Xianglong Liu. 2023. Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling. arXiv preprint arXiv:2304.09145

    2023 arXiv

  51. [59]

    Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, and Xianglong Liu. 2022. Outlier suppression: Pushing the limit of low-bit transformer language models. Advances in Neural Information Processing Systems, 35:17402--17414

    2022

  52. [60]

    Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023 a . A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122--1136

    2023

  53. [61]

    Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Zhewei Yao, and Yuxiong He. 2023 b . Understanding int4 quantization for transformer models: Latency speedup, composability, and failure cases. arXiv preprint arXiv:2301.12017

    2023 arXiv

  54. [62]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  55. [63]

    Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhensu Chen, Xiaopeng Zhang, and Qi Tian. 2023. Qa-lora: Quantization-aware low-rank adaptation of large language models. arXiv preprint arXiv:2309.14717

    2023 arXiv

  56. [64]

    Jiaming Yang, Chenwei Tang, Caiyang Yu, and Jiancheng Lv. 2024. Gwq: Group-wise quantization framework for neural networks. In Asian Conference on Machine Learning, pages 1526--1541. PMLR

    2024

  57. [65]

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168--27183

    2022

  58. [66]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  59. [67]

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. 2016. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160

    2016 arXiv

  60. [68]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  61. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.