Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MambaQuant is a post-training quantization framework that keeps Mamba-based vision and language models within 1% of full precision at 8-bit weights and activations.

desk verdict Solid KLT-based rotation idea for Mamba PTQ, but test-set calibration and an unproven online smoothing step undercut the reported numbers. read the letter →

arxiv 2501.13484 v3 pith:3ZGNVBKB submitted 2025-01-23 cs.LG cs.AIcs.CL

Zukang Xu , Yuxuan Yue , Xing Hu , Zhihang Yuan , Zixu Jiang , Zhixuan Chen , Jiangyong Yu , Chen Xu
show 2 more authors
Sifan Zhou Dawei Yang
This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords Mambaquantizationpost-trainingstatespacemodelsHadamardrotationKarhunen-Loevetransformchannelvariancealignmentparallelscan8-bit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MambaQuant is a post-training quantization framework built for Mamba, the selective state-space sequence model. The paper argues that Mamba's gate projections, output projections, and matrix multiplications contain strong channel outliers, and that the parallel-scan operator amplifies those outliers into heavy-tailed distributions that ordinary Hadamard rotation cannot fix. MambaQuant's fix is two variance-aligned rotations: a Karhunen-Loève-enhanced rotation for offline layers and a smooth-fused rotation for online layers. If the paper is right, Mamba models can be deployed with 8-bit weights and activations at less than 1% accuracy loss on vision and language tasks, without retraining.

What carries the argument

The central object is the variance-aligned rotation matrix. In offline mode it is $H_K = K H$, where $K$ is the eigenvector matrix of the calibration covariance $C_X$ and $H$ is a Hadamard matrix; this makes the channel variances of the rotated data equal to the average of all eigenvalues, while preserving Hadamard's ability to balance maximum values. In online mode the machinery is a smoothing vector applied before the Hadamard transform: the paper replaces SiLU with S-SiLU, $x \odot \sigma(s \odot x)$, and absorbs the smoothing factors into the gate, output, C, and B projection weights, while the first token of $\Delta$ is modified by $\operatorname{addcmul}(-\ln(s_{\mathrm{mm}}), \Delta(1), A) = A\Delta(1) - \ln(s_{\mathrm{mm}})$ to propagate the factor through the parallel scan. These two mechanisms together are what make the quantization distributions uniform.

What would settle it

Compute the recurrence on one Mamba block in floating point: run the parallel scan with the original parameters and scale the result by the smoothing vector; then run it again changing only the first token of $\Delta$ as in Equation 16 and leaving everything else fixed. If the two outputs differ by more than rounding error, the online smoothing does not implement the claimed propagation and the online component of MambaQuant loses its stated mechanism.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that channel variance, not just channel maximum, is the obstacle to quantizing Mamba, and that making channel variances equal makes 8-bit quantization nearly lossless. The offline construction multiplies the Hadamard matrix by the eigenvector matrix $K$ of the calibration covariance, forming $H_K = K H$, and the paper shows the diagonal of the transformed covariance becomes $(1/((n-1)m))\sum_j \lambda_j$, identical for every channel. The online construction applies a smoothing vector before the Hadamard rotation, absorbing the smoothing into adjacent weights: through S-SiLU for the output projection and through the B projection plus a first-token change to $\Delta$ for the matrix multiplication. The reported result is W8A8 accuracy within 1 point of floating-point baselines across Vim, Mamba-ND, and Mamba-LLM, and roughly 1 point loss at W4A8 for vision, with larger gaps for language.

Load-bearing premise

The load-bearing premise is that a per-channel smoothing factor on the output of the parallel scan can be realized by changing only the first token of $\Delta$, as Equation 16 states, with no derivation given in the paper.

Editorial extensions

If this is right

  • Mamba models can be served at W8A8 with near-full accuracy, which reduces activation memory and enables faster integer inference.
  • The failure of Hadamard-only rotation on Mamba is diagnosed as a variance mismatch, so quantization work on state-space models should be evaluated on channel-variance alignment, not only outlier range.
  • The offline KLT-enhanced rotation is built from calibration data and fused into weights, so it adds no inference-time cost for the layers it covers.
  • The online smooth-fused rotation keeps overhead small: the paper reports about 0.01% more parameters and 0.91% more FLOPs on Mamba-2.8B.
  • W4A8 remains a harder regime: the paper reports about 1% loss on vision and a larger drop on language, so the '<1%' claim is specific to W8A8.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the paper's Appendix A.1 concedes that 4-bit weights and an online KLT are not solved, the central '<1% loss' claim should be read as scoped to 8-bit weights and activations.
  • Editorial inference: the addcmul trick, if sound, is a general recipe for folding per-channel scales into exponential recurrent states by adjusting only the initial decay, which would apply to other selective state-space models.
  • Editorial inference: a natural stress test is to push the calibration-derived KLT matrix on inputs far outside the calibration distribution; the paper's Figure 12 suggests it remains no worse than Hadamard, but the failure boundary is not established.
  • Editorial inference: computing a combined KLT-plus-Hadamard rotation for the online path is the most direct route toward closing the W4A8 gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MambaQuant, a post-training quantization (PTQ) framework for Mamba-family models. In offline mode, it estimates a per-layer Karhunen-Loève transformation (KLT) matrix from calibration data and combines it with a Hadamard rotation to equalize channel variances before quantization. In online mode, it introduces a smooth-fused rotation that applies per-channel smoothing to the gate/output projections and to the PScan-output matmul, with the smoothing scales folded into neighboring weights. Experiments on Vim, Mamba-ND, and Mamba-LLM report W8A8 accuracy within 1 point of FP16 and W4A8 improvements over RTN, GPTQ+RTN, SmoothQuant, and QuaRot. The abstract's central claim is that W8A8 quantization achieves less than 1% accuracy loss on Mamba-based vision and language tasks.

Significance. The paper identifies a real and previously underappreciated problem: Hadamard rotation alone does not align channel variances in Mamba layers, and the PScan operator amplifies outliers. The KLT variance derivation in Eqs. (8)-(13) is mathematically sound for the calibration distribution, and the paper ships code. If the W8A8 results survive a clean held-out evaluation, MambaQuant would be a useful baseline for Mamba PTQ. However, the current empirical support is weakened by the calibration protocol and by an underived smoothing-propagation step, so the significance claim rests on corrections to those points.

major comments (3)
  1. [Section 5, Models and datasets; Appendix A.7] The calibration protocol invalidates the central empirical claim as currently presented. Vision calibration is explicitly drawn from the ImageNet and UCF-101 test sets, and Appendix A.7 states that the Table 2 language experiments are calibrated on HellaSwag, which is itself one of the five datasets averaged in Table 2. Because the KLT matrix and smoothing scales are fitted to the evaluation distribution, the reported W8A8 'less than 1%' gaps are not a valid measure of generalization. Please re-run the experiments with calibration data from a training or otherwise disjoint held-out split, and separately report calibration and non-calibration accuracy; until then the abstract's central claim is not supported by the tables.
  2. [Section 4.3, Eq. (16)] The propagation of the smoothing factor through the PScan recurrence is asserted rather than derived. The equation addcmul(-ln(s_mm), Delta(1), A) = A*Delta(1) - ln(s_mm) modifies only the first token, but a uniform scaling of the PScan output by s_mm would be implemented by scaling B (and the initial state) in h(t) = A h(t-1) + B x(t), not by changing the state transition at t=1. Since Eq. (4) defines the transition as A = e^{A_bar ⊙ Δ}, altering Delta(1) changes the effective A only at the first step. Please provide an exact derivation of the equivalence, or state that this is an approximation and quantify the induced error; without this, the online smooth-fused rotation for the matmul layer is not established.
  3. [Table 2, W8A8 row; Section 5.1] Mamba-790m drops from 54.8 to 53.8 under W8A8, a 1.0 percentage-point loss, which contradicts the abstract's 'less than 1% accuracy loss' if the metric is percentage points. Section 5.1 says 'within 1 points' for Table 1, showing a percentage/point terminological inconsistency. Please state the metric explicitly and correct the claim or the table. The same issue affects the introductory statement that 4-bit weights give 'about 1%' loss on vision tasks, since Table 1 shows losses of 4.0 and 4.6 points on Vim-T and Vim-T†.
minor comments (5)
  1. [Throughout] There are several typos that should be corrected: 'Sofplus' in Eq. (3), 'Euation' near Eq. (16), 'desined' in Appendix A.1, and 'constrain' in Section 4.2.
  2. [Table 4] Table 4 is difficult to read because the W8A8 and W4A8 panels are interleaved; for Mamba-790M the FP16 row lists both 54.6 and 58.6, which should be separated and clearly labeled.
  3. [Table 6, Appendix A.6] The header placement, with 'Avg ACC' appearing before the per-dataset columns, makes the column alignment ambiguous; please reformat the table so each column is clearly identified.
  4. [Abstract and Section 1] The spelling 'Quarot' is used inconsistently with 'QuaRot' elsewhere in the paper; please unify the spelling.
  5. [Appendix A.7] The generalization analysis provides only distribution visualizations for non-calibration inputs; without quantitative accuracy numbers on disjoint data, the generalization claim remains qualitative.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: KLT and smooth-fused rotations are constructed to equalize calibration-data variance by design, and the empirical accuracy claims rest on test-set calibration leakage rather than on a derivation that reduces to its own inputs.

full rationale

The claimed derivation chain is not circular. The KLT-enhanced rotation is constructed from the calibration-data covariance (Eq. 10), and Eq. 13 shows that, on that same calibration data, the rotated matrix has exactly uniform channel variance. This is a mathematical identity, not a prediction; the paper separately argues generalization in Appendix A.7. The smooth-fused rotation is an exact weight reparameterization in Eq. 15 (W'_g = W_g / s_out, W'_o = s_out * W_o), and Eq. 16 gives a proposed propagation rule for the smoothing factor through the PScan; whether Eq. 16 is valid is a correctness question, not a circular one. Self-citations (e.g., S-SiLU attributed to Hu et al. 2024, with overlapping authorship) are explicit definitions or related-work attributions and are not load-bearing. The main circularity-adjacent concern is the evaluation protocol, not the derivation: Section 5 calibrates vision quantization on 128-image samples from the ImageNet and UCF-101 test sets, and Appendix A.7 states that the Table 2 language results are calibrated on HellaSwag, which is included in the reported five-dataset average. That is test-set leakage and should be weighed as a correctness and validity risk, but the reported accuracies are not equal to the calibration inputs by construction, so it does not make the method's derivation circular. Score 1 reflects only the minor self-citation and the absence of any stronger circular step.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method rests on two data-fitted quantities (KLT matrix and smoothing vectors), four domain assumptions about Mamba's structure and calibration representativeness, and one underspecified PScan propagation step. No new physical entities are introduced.

free parameters (3)
  • KLT eigenvector matrix K per transformed layer = eigenvectors of calibration covariance for that layer's activations
    Equations 10-11: K is computed from calibration data and defines the offline rotation. It is data-dependent and is the main fitted object of the offline mode.
  • smoothing vector s_out for output projection = not specified in paper
    Section 4.3: per-channel smoothing factor absorbed into S-SiLU and output projection weights; the paper does not give the optimization or heuristic used to set it.
  • smoothing vector s_mm for matmul scan output = not specified in paper
    Section 4.3: per-channel factor propagated through B and delta; selection criterion not stated.
assumptions (5)
  • domain assumption Columns of the data matrices used for KLT are approximately zero-mean.
    Section 4.2 states that the mean value for each channel of the Mamba weights and activations is typically close to zero, meeting the applicable conditions of KLT. If this fails, the covariance-based KLT does not equalize the actual channel variances.
  • standard math An orthogonal rotation can be absorbed into adjacent linear weights without changing the network function.
    This is the QuaRot equivalence property (Sections 4.2-4.3). It is true for linear layers, but the paper assumes it remains valid across Mamba's nonlinear gates.
  • domain assumption Calibration data is representative of test inputs.
    The KLT matrix and smoothing factors are fit on 128 ImageNet test images for vision or HellaSwag for language and then applied to all evaluation data (Section 5, Appendix A.7).
  • ad hoc to paper The smoothing factor can be pushed through the parallel scan recurrence by modifying B and first-token delta.
    Section 4.3, Eq. 16 introduces addcmul on Delta(1) without derivation; used to justify online matmul smoothing.
  • domain assumption The identified hard layers (gate projection, output projection, matrix multiplication) are the only layers requiring special handling.
    Section 4.1 bases the design on outliers in these layers; if other layers have comparable outliers, the method would miss them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods." pith.science (2026). https://pith.science/paper/3ZGNVBKB

@misc{pith2026250113484,
  author       = {Pith},
  title        = {Pith review of: MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ZGNVBKB}},
  note         = {Machine review of arXiv:2501.13484}
}
abstract

Mamba is an efficient sequence model that rivals Transformers and demonstrates significant potential as a foundational architecture for various tasks. Quantization is commonly used in neural networks to reduce model size and computational latency. However, applying quantization to Mamba remains underexplored, and existing quantization methods, which have been effective for CNN and Transformer models, appear inadequate for Mamba models (e.g., Quarot suffers a 21% accuracy drop on Vim-T$^\dagger$ even under W8A8). We have pioneered the exploration of this issue and identified several key challenges. First, significant outliers are present in gate projections, output projections, and matrix multiplications. Second, Mamba's unique parallel scan further amplifies these outliers, leading to uneven and heavy-tailed data distributions. Third, even with the application of the Hadamard transform, the variance across channels in weights and activations still remains inconsistent. To these ends, we propose MambaQuant, a post-training quantization (PTQ) framework consisting of: 1) Karhunen-Loeve Transformation (KLT) enhanced rotation, rendering the rotation matrix adaptable to diverse channel distributions. 2) Smooth-Fused rotation, which equalizes channel variances and can merge additional parameters into model weights. Experiments show that MambaQuant can quantize both weights and activations into 8-bit with less than 1% accuracy loss for Mamba-based vision and language tasks. To the best of our knowledge, MambaQuant is the first comprehensive PTQ design for the Mamba family, paving the way for further advancements in its application.

Figures

Figures reproduced from arXiv: 2501.13484 by the authors.

Figure 1
Figure 1. Visualized distribution of hard layers for Mamba quantization. (a) denotes the weight of the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Maximum values (blue color) and variances (red color) distribution across channels of: (a) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Mamba block architecture. The state space models (SSMs) are typically re￾garded as contiguous linear time-invariant (LTI) systems (Kalman, 1960), which map an input sig￾nal x(t) ∈ R to its output y(t) ∈ R through a hidden state h(t) ∈ R d×1 : h(t) = Ah(t − 1) + Bx(t), (1) yssm(t) = Ch(t) + Dx(t), (2) where A ∈ R d×d , B ∈ R d×1 , C ∈ R 1×d , D ∈ R 1×1 are weighting parameters, t ∈ Z +, and h(0) is an initial hidden … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Offline transformation designs utilizing the KLT-Enhanced rotation. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Fusing smooth parameters into the Mamba structure. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Online transfor￾mation designs utilizing the smooth-fused rotation. After smoothing, the channel variances of activations for the output pro￾jection and the matrix multiplication becomes relatively uniform. Subse￾quently, we modify and apply the online Hadamard rotatio…
Figure 7
Figure 7. Figure 7: Data of activatin of gate projection distribution and quantization of losses. (a) Original data [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Data of weight of output projection distribution and quantization of losses. (a) Original [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Variances of input of output projection layer blocks in Vim-T: unequal across channels pre [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Variances of input of gate projection layer blocks in Mamba-790m: unequal across [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Variances of weight of gate projection layer blocks in Mamba-790m: unequal across [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Analysis of Activation Values in the in projection of the block 0 of the Vim-T model.(a) Display using 384 images from ImageNet for calibration. (b), (c) and (d) Display of other data outside the calibration set [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: The amplification phenomenon of activations by the Pscan operator in different blocks of [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing

    cs.LG 2025-08 conditional novelty 6.0 of 10

    An end-to-end Mamba edge accelerator using hardware-friendly approximations, INT8 quantization, and NAS achieves 4.95x-5.62x lower latency and 1.63x-19.9x smaller models than ViT/CNN baselines.

Reference graph

Works this paper leans on

46 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [2]

    Slicegpt: Compress large language models by deleting rows and columns

    Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. Slicegpt: Compress large language models by deleting rows and columns. International Conference on Learning Representations (ICLR), 2024 a

    work page 2024

  2. [3]

    Quarot: Outlier-free 4-bit inference in rotated llms

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated llms. arXiv preprint arXiv:2404.00456, 2024 b

    arXiv 2024

  3. [4]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Association for the Advancement of Artificial Intelligence, 2020

    work page 2020

  4. [5]

    A systematic classification of knowledge, reasoning, and context within the arc dataset

    Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue-Nkoutche, Pavan Kapanipathi, Nicholas Mattei, et al. A systematic classification of knowledge, reasoning, and context within the arc dataset. arXiv preprint arXiv:1806.00358, 2018

    arXiv 2018

  5. [6]

    Quip: 2-bit quantization of large language models with guarantees

    Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. Quip: 2-bit quantization of large language models with guarantees. Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  6. [7]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    arXiv 2018

  7. [8]

    Karhunen-loeve transform

    R Dony et al. Karhunen-loeve transform. The transform and data compression handbook, 1 0 (1-34): 0 29, 2001

    work page 2001

  8. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

Show all 46 references
  1. [10]

    Model quantization and hardware acceleration for vision transformers: A comprehensive survey

    Dayou Du, Gu Gong, and Xiaowen Chu. Model quantization and hardware acceleration for vision transformers: A comprehensive survey. arXiv preprint arXiv:2405.00314, 2024

    2024 arXiv

  2. [11]

    Unified matrix treatment of the fast walsh-hadamard transform

    Fino and Algazi. Unified matrix treatment of the fast walsh-hadamard transform. IEEE Transactions on Computers, 100 0 (11): 0 1142--1146, 1976

    1976

  3. [12]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  4. [13]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  5. [14]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher R \'e . Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021

    2021 arXiv

  6. [15]

    I-llm: Efficient integer-only inference for fully-quantized low-bit large language models

    Xing Hu, Yuan Chen, Dawei Yang, Sifan Zhou, Zhihang Yuan, Jiangyong Yu, and Chen Xu. I-llm: Efficient integer-only inference for fully-quantized low-bit large language models. arXiv preprint arXiv:2405.17849, 2024

    2024 arXiv

  7. [16]

    Localmamba: Visual state space model with windowed selective scan

    Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, and Chang Xu. Localmamba: Visual state space model with windowed selective scan. arXiv preprint arXiv:2403.09338, 2024

    2024 arXiv

  8. [17]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision an...

    2018

  9. [18]

    A new approach to linear filtering and prediction problems

    Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82 0 (1): 0 35--45, 1960

    1960

  10. [19]

    Quantizing deep convolutional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018

    2018 arXiv

  11. [20]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  12. [21]

    Mamba-nd: Selective state space modeling for multi-dimensional data

    Shufan Li, Harkanwar Singh, and Aditya Grover. Mamba-nd: Selective state space modeling for multi-dimensional data. arXiv preprint arXiv:2402.05892, 2024

    2024 arXiv

  13. [22]

    Repq-vit: Scale reparameterization for post-training quantization of vision transformers

    Zhikai Li, Junrui Xiao, Lianwei Yang, and Qingyi Gu. Repq-vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 17227--17236, 2023

    2023

  14. [23]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978, 2023

    2023 arXiv

  15. [24]

    Fq-vit: Post-training quantization for fully quantized vision transformer

    Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824, 2021

    2021 arXiv

  16. [25]

    Pd-quant: Post-training quantization based on prediction difference metric

    Jiawei Liu, Lin Niu, Zhihang Yuan, Dawei Yang, Xinggang Wang, and Wenyu Liu. Pd-quant: Post-training quantization based on prediction difference metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 24427--24437, 2023

    2023

  17. [26]

    Vmamba: Visual state space model

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166, 2024

    2024 arXiv

  18. [27]

    torch.addcmul

    PyTorch. torch.addcmul. https://pytorch.org/docs/stable/generated/torch.addcmul.html, 2023. 2024.09.24

    2023

  19. [28]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 0 211--252, 2015

    2015

  20. [29]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 2021

    2021

  21. [30]

    Omniquant: Omnidirectionally calibrated quantization for large language models

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models. CoRR, abs/2308.13137, 2023

    2023 arXiv

  22. [31]

    Simplified state space layers for sequence modeling

    Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling. arXiv preprint arXiv:2208.04933, 2022

    2022 arXiv

  23. [32]

    A dataset of 101 human action classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. A dataset of 101 human action classes from videos in the wild. Center for Research in Computer Vision, 2 0 (11): 0 1--7, 2012

    2012

  24. [33]

    Quip\#: Even better llm quantization with hadamard incoherence and lattice codebooks

    Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. Quip\#: Even better llm quantization with hadamard incoherence and lattice codebooks. Forty-first International Conference on Machine Learning, 2024

    2024

  25. [34]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 2017

    2017

  26. [35]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. arXiv preprint arXiv:2211.10438, 2022

    2022 arXiv

  27. [36]

    Visual mamba: A survey and new outlooks, 2024

    Rui Xu, Shu Yang, Yihui Wang, Yu Cai, Bo Du, and Hao Chen. Visual mamba: A survey and new outlooks, 2024

    2024

  28. [37]

    An efficient multi-task learning cnn for driver attention monitoring

    Dawei Yang, Yan Wang, Ran Wei, Jiapeng Guan, Xiaohua Huang, Wei Cai, and Zhe Jiang. An efficient multi-task learning cnn for driver attention monitoring. Journal of Systems Architecture, 148: 0 103085, 2024

    2024

  29. [38]

    LLMViewer

    Zhihang Yuan. LLMViewer . https://github.com/hahnyuan/LLMViewer, 2024. Accessed: February 14, 2024

    2024

  30. [39]

    Wkvquant: Quantizing weight and key/value cache for large language models gains more

    Yuxuan Yue, Zhihang Yuan, Haojie Duanmu, Sifan Zhou, Jianlong Wu, and Liqiang Nie. Wkvquant: Quantizing weight and key/value cache for large language models gains more. arXiv preprint arXiv:2402.12065, 2024

    2024 arXiv

  31. [40]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  32. [41]

    Cobra: Extending mamba to multi-modal large language model for efficient inference

    Han Zhao, Min Zhang, Wei Zhao, Pengxiang Ding, Siteng Huang, and Donglin Wang. Cobra: Extending mamba to multi-modal large language model for efficient inference. arXiv preprint arXiv:2403.14520, 2024

    2024 arXiv

  33. [42]

    A survey on efficient inference for large language models

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, et al. A survey on efficient inference for large language models. arXiv preprint arXiv:2404.14294, 2024

    2024 arXiv

  34. [43]

    Vision mamba: Efficient visual representation learning with bidirectional state space model

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024

    2024 arXiv

  35. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  36. [45]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  37. [46]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  38. [47]

    " @ K w?O ?+[.-m >2O eh> sqǙ)Sd ӌ*hL /RM vk

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.