QMamba: Post-Training Quantization for Vision State Space Models

Jiacheng Li; Ruikang Xu; Xiaoyu Liu; Yinda Chen; Yinglong Li; Zhiwei Xiong

REVIEW 4 major objections 5 minor 1 cited by

QMamba: Post-Training Quantization for Vision State Space Models

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QMamba is a post-training quantization scheme that custom-tailors quantizers to Mamba's state-space operators, and it claims large accuracy gains at 4-bit activations.

desk verdict Solid PTQ method for vision SSMs with a sensible design and good ablations, but the QDrop baseline fidelity behind the 21-point headline is ambiguous and needs verification. read the letter →

arxiv 2501.13624 v1 pith:FHE2NRMQ submitted 2025-01-23 cs.CV

Yinglong Li , Xiaoyu Liu , Jiacheng Li , Ruikang Xu , Yinda Chen , Zhiwei Xiong This is my paper · ORCID

classification cs.CV

keywords post-trainingquantizationstatespacemodelsMambavisionSSMsactivationlog2temporalgroupingedgedeployment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that general-purpose post-training quantizers fail on vision state space models because two SSM activations have unusual distributions: the discrete parameters $A_t$ concentrate densely near 1 with a long sparse tail, and the hidden states $h_t$ change their value ranges dramatically across time steps. QMamba is designed around these two facts: LtSQ applies log2-based non-uniform quantization to $A_t$ so the dense region near 1 gets fine intervals, and TGQ splits the hidden-state sequence into temporal groups, each with its own scaling factor, so a single scale does not have to fit all time steps. The result is a PTQ framework that outperforms existing PTQ methods consistently on Vim and VMamba, with the headline being a 21.0-point Top-1 accuracy gain over QDrop on Vim-S with 6-bit weights and 4-bit activations. If these distributional patterns generalize, QMamba offers a template for low-bit deployment of selective SSMs on edge hardware.

What carries the argument

The paper's machinery is the state equation $h_t = A_t h_{t-1} + B_t x_t$ of a selective SSM. LtSQ (Long-tailed Skewness Quantization) checks a skewness condition $\mathrm{Median}(A_t) > \alpha$ on calibration data, and if satisfied, maps $A_t$ through a $b$-bit log2 quantizer as $\hat{A}_t = 1 - 2^{-A_t^q}$, putting small intervals on the dense values near 1 and producing a hardware-friendly bit-shift for the $A_t h_{t-1}$ product. TGQ (Temporal Group Quantization) splits the $L$ hidden-state vectors along the time dimension into groups of length $\lambda$, and each group receives its own tensor-wise uniform quantizer with its own scale and zero-point, so the quantizer adapts to how the range of $h_t$ expands and contracts along the sequence. Both are wrapped in a QDrop-style block-wise reconstruction pipeline that initializes scales from 1st/99th percentiles and fine-tunes them with MSE loss.

What would settle it

Take an SSM-based vision model whose $A_t$ activations have a median well below 0.9 across most layers (so LtSQ nearly always falls back to uniform) and whose hidden-state ranges are roughly constant in time; if QMamba's W6A4 accuracy advantage over QDrop does not shrink to near zero, then the distributional story is not the source of the gains.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the quantization bottleneck in vision state space models lies in two tensors inside the SSM recurrence, $A_t$ and $h_t$, and that both can be quantized far more accurately by exploiting their structure than by uniform tensor-wise scaling. For $A_t$ it shows that a log2-style quantizer (LtSQ) on the quantity $1 - A_t$ gives fine resolution exactly where the values cluster, and because the de-quantized value is a power of two, the multiply $A_t h_{t-1}$ becomes an integer bit-shift. For $h_t$ it shows that dividing the sequence into groups of ten time steps and assigning each group its own scaling factor (TGQ) tracks the changing ranges closely enough to cut quantization error. The paper's evidence is the comparison table across three sizes each of Vim and VMamba at W8A8, W6A6, and W6A4, where QMamba reports the best Top-1 accuracy in every configuration, and its ablations confirm both components contribute, with LtSQ contributing most on Vim-S.

Load-bearing premise

The design rests on the premise that every targeted SSM will show the same two distributional signatures — $A_t$ long-tailed and skewed with median near 1, and $h_t$ ranges that vary strongly over time — and that the fixed thresholds $\alpha=0.9$ and $\lambda=10$ chosen on Vim and VMamba will suit other models.

Editorial extensions

If this is right

At 4-bit activations, Vim-S goes from 24.9% Top-1 accuracy under QDrop to 45.9% under QMamba, a 21.0-point gain, suggesting the largest wins come exactly where uniform quantizers collapse.
The bit-shift reformulation of $A_t h_{t-1}$ means the extra accuracy costs almost no extra compute, so quantized SSMs remain deployment-friendly on edge devices.
The same two module designs can be dropped into any Mamba-style block, so models like VMamba benefit even though their 2D scan structure differs from Vim's.
Scaling-factor initialization matters more for SSMs than for CNNs and ViTs: percentile initialization beats MinMax and OMSE at low bit widths on these models, per the paper's Table 2.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the distributional observations transfer to Mamba-style models in other modalities (language, audio, video), LtSQ and TGQ could give similar gains there; nothing in this paper tests that transfer, so it is a prediction, not a result.
The fixed hyperparameters $\alpha=0.9$ and $\lambda=10$ were tuned on Vim and VMamba with ImageNet; a natural extension is to make them adaptive per layer or per model, since the skewness check exists precisely to detect when logs should not be applied.
Because LtSQ falls back to uniform quantization whenever a tensor's median is below $\alpha$, one could measure how often the switch triggers across layers of a new architecture and use that fraction as a quick diagnostic of how much of QMamba's gain will materialize there.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, this is a competently motivated PTQ paper for vision SSMs, and the two custom quantizers—LtSQ's log-domain treatment of A_t and TGQ's temporal grouping of h_t—are sensible and well-ablated. Second, the headline 21-point gain over QDrop rests on a baseline whose implementation is ambiguous, so treat that number as unverified until the authors clarify or ship code.

What's genuinely useful: the distributional observations (long-tailed A_t, time-varying h_t) are visualized and backed by sensitivity experiments; the design follows directly from those observations. The ablations in Table 3 show both components help, and Table 4 shows the alpha/lambda choices are not knife-edge, though they are tuned on the same validation set. The bit-shift formulation for A_t * h_{t-1} is a nice practical touch. On the evidence in the paper, QMamba does outperform the stated baselines on Vim and VMamba at W6A4 and W6A6.

The soft spots, in order of severity. (1) QDrop baseline fidelity. Section 4.1 says only 'we follow the framework of QDrop,' and the method section describes plain block-wise MSE fine-tuning with no mention of QDrop's random activation-drop mechanism. Table 3's baseline row is described as 'QDrop as the baseline without our customized LtSQ and TGQ' with all scaling factors initialized and finetuned with the same settings—which sounds like their own pipeline, not necessarily the published QDrop. Without code, I can't tell whether the 24.9% baseline is official QDrop or a weakened reimplementation. That gap sits directly under the abstract's 21.0% claim. (2) No code, no error bars. Single-seed results on one calibration split make the gains hard to assess. (3) Hyperparameters alpha and lambda are chosen on the ImageNet validation set; this is common in PTQ papers but should be acknowledged as selection, not as fixed universal constants. (4) 'One of the first PTQ frameworks for SSMs' is not checked against concurrent work like Quamba or MambaQuant; at least one of those may predate this arXiv posting.

None of this is fatal to the core idea. The distributional analysis and the quantizer designs are likely right, and the experiments are internally consistent. But the paper is not fully verifiable as submitted.

Who it's for: anyone doing quantization or efficient deployment of Mamba-family vision models. It deserves a serious referee, and I'd ask the authors for code, a precise description of the QDrop baseline (ideally comparing against official QDrop with and without random dropping), and error bars before accepting.

Referee Report

4 major / 5 minor

Summary. The paper proposes QMamba, a post-training quantization framework for SSM-based vision models such as Vim and VMamba. Based on an analysis of activation distributions, it identifies two challenges: the discrete parameters A_t are long-tailed and skewed, and the hidden states h_t have highly time-varying ranges. The paper introduces Long-tailed Skewness Quantization (LtSQ), a log2-based non-uniform quantizer for A_t, and Temporal Group Quantization (TGQ), which applies separate uniform scaling factors to groups of h_t along the time dimension. Scaling factors are initialized with percentile statistics and fine-tuned block-wise on a calibration set using an MSE loss. Experiments on ImageNet across Vim-T/S/B and VMamba-T/S/B at W8A8, W6A6, and W6A4 report consistent improvements over MinMax, Percentile, OMSE, AdaRound, BRECQ, and QDrop, including a headline 21.0% Top-1 improvement over QDrop on Vim-S at W6A4.

Significance. If the reported results hold, QMamba is a useful and timely contribution: it is among the first PTQ methods specifically designed for SSM operators, and it identifies distributional properties that plausibly matter for quantization accuracy. The proposed LtSQ and TGQ are simple and well motivated by the visualized statistics, and the efficiency analysis shows a meaningful storage/FLOPs trade-off. The main results are internally consistent and the ablations demonstrate the individual contributions of LtSQ and TGQ. However, the significance is moderated by gaps in experimental rigor: the fidelity of the QDrop baseline is not explicitly established, hyperparameters are selected on the evaluation benchmark, and no variance information is reported over calibration sets or optimization runs.

major comments (4)

[§3.5, §4.1, Table 1] The headline claim of a 21.0% Top-1 improvement over QDrop on Vim-S (W6A4) depends on the QDrop baseline faithfully reproducing the published QDrop algorithm. The paper says in §4.1 that it follows the framework of QDrop, but §3.5 describes only block-wise MSE fine-tuning of scaling factors and never mentions QDrop's defining random-dropping of activation quantization during reconstruction. If the QDrop rows in Tables 1 and 3 correspond to a uniform-quantizer reimplementation without random dropping, the comparison would inflate QMamba's advantage. Please state explicitly whether the baseline uses the published QDrop procedure, and ideally provide code or a reference implementation to make the reported baseline numbers reproducible.
[§5.2, §5.3, Table 4] The hyperparameters alpha=0.9 and lambda=10 are selected by sweeping on the same evaluation benchmark on which the main results are reported (Vim-B and VMamba-B at W6A4 in Table 4). This is a form of test-set selection that makes the reported gains optimistic. The authors should either use a held-out validation split for hyperparameter selection, report results across the full sweep in the main tables, or discuss the sensitivity of the final accuracy to these choices more carefully. This point is load-bearing because the claim that QMamba consistently outperforms baselines relies on having chosen favorable hyperparameter values.
[§4.1 and Tables 1–4] No variance or error-bar information is reported. The calibration set consists of 1024 randomly sampled images, and the block-wise fine-tuning is stochastic (random calibration batches, Adam with random initialization of learnable scales is not mentioned but the process is inherently noisy). Some comparisons in Table 1, such as VMamba-B at W6A6 (82.0 vs. 82.1), are within the range of plausible calibration-set noise. Please report the mean and standard deviation over at least three independent calibration samples or seeds, or otherwise justify that the reported differences are statistically meaningful.
[§3.2, §3.3, §3.4] The two design motivations are established empirically on Vim and VMamba with ImageNet inputs only. The paper claims a general framework for vision SSMs, but the fixed skewness boundary alpha=0.9 and group length lambda=10 may not transfer to other SSM variants, sequence lengths, or tasks. If the scope is intended to be broader than the tested models, please add experiments or analysis on at least one additional SSM architecture or task (e.g., detection, segmentation, or a different SSM initialization) to support the generality claim. Otherwise, limit the claim to the evaluated settings.

minor comments (5)

[§5.2] The sentence 'we conduct experiments with different skewness boundaries α by setting λ = 0 .9' appears to be a typo: λ is the group length, not the skewness boundary. It should read 'setting α = 0.9' or 'setting λ = 10' to match Table 4.
[Abstract and §1] The name 'Mamba' is misspelled as 'Mamaba' in the abstract and in the first sentence of the introduction.
[§3.3, Eq. (6)] Equation (6) implicitly assumes A_t lies in (0,1) so that 1-A_t is positive. Please state the domain of A_t and discuss what happens if some A_t values are outside this range, because the log2 operation is undefined for non-positive arguments.
[§3.5, Eq. (8)] The bit-shift implementation in Eq. (8) needs a note about signed values; right-shifting a possibly negative integer is not a standard portable operation. Clarify how negative hidden states are handled.
[§4.1] The implementation details do not state whether weight quantization uses per-tensor or per-channel scaling factors. Since the paper says tensor-wise quantization for weights in §3.1, please confirm this in the experimental setup and discuss any implications for the comparison with AdaRound/BRECQ, which typically use per-channel or more fine-grained weight quantizers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: QMamba is an empirical PTQ construction evaluated on an external benchmark, with no self-citation chain and no prediction that reduces to its inputs.

full rationale

The paper's derivation chain is observational and empirical, not definitional. Section 3.2 observes distributional properties of A_t and h_t; Sections 3.3-3.4 design LtSQ and TGQ to match those properties; Section 4 measures accuracy on ImageNet against external baselines. The reported 21.0% improvement (Sec. 4.2, Table 1) is a measured comparison, not a quantity fitted or derived from the method's own equations. No load-bearing step invokes the authors' own prior work: the reference list contains no self-citations, and the cited QDrop/BRECQ/PTQ4ViT/RepQViT results are external methods used as baselines or inspiration. The hyperparameters alpha=0.9 and lambda=10 are selected via ablations on the same benchmark (Table 4), which is mild tuning rather than circularity; the paper does not relabel this tuning as a prediction. The skeptic's concern about whether the QDrop baseline faithfully includes random activation dropping is a verification/correctness gap in the comparison, not a circularity of the derivation. Overall, no claimed result is equivalent to its own input by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two empirical distributional observations about A_t and h_t, which are presented as universal properties but are shown only on two model families on ImageNet. The hand-tuned hyperparameters alpha and lambda are free parameters selected on the evaluation set. No new physical or conceptual entities are introduced; the method is a composition of known quantization primitives.

free parameters (2)

alpha (skewness boundary) = 0.9
Threshold for deciding when to use LtSQ; selected via ablation on ImageNet validation (Table 4).
lambda (group length) = 10
Number of time steps per quantization group in TGQ; selected via ablation on ImageNet validation (Table 4).

assumptions (4)

domain assumption The discrete parameter A_t exhibits long-tailed skewness with median near 1 across SSM-based vision models.
Observed on Vim and VMamba on ImageNet; the method assumes this generalizes to other SSM variants and tasks.
domain assumption The hidden state h_t has highly dynamic distribution ranges across time steps.
Observed on the same models; motivates temporal grouping that is central to TGQ.
domain assumption A calibration set of 1024 randomly sampled images is representative enough to learn quantization scaling factors.
Standard PTQ assumption; the paper does not vary the sampling seed or test calibration set size sensitivity.
standard math The bit-shift operation in Eq. 8 correctly implements the multiplication of quantized A_t and h_{t-1}.
Follows from log2 quantization and integer arithmetic; routine for 2's complement shift operations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of QMamba: Post-Training Quantization for Vision State Space Models." pith.science (2026). https://pith.science/paper/FHE2NRMQ

@misc{pith2026250113624,
  author       = {Pith},
  title        = {Pith review of: QMamba: Post-Training Quantization for Vision State Space Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FHE2NRMQ}},
  note         = {Machine review of arXiv:2501.13624}
}

read the original abstract

State Space Models (SSMs), as key components of Mamaba, have gained increasing attention for vision models recently, thanks to their efficient long sequence modeling capability. Given the computational cost of deploying SSMs on resource-limited edge devices, Post-Training Quantization (PTQ) is a technique with the potential for efficient deployment of SSMs. In this work, we propose QMamba, one of the first PTQ frameworks to our knowledge, designed for vision SSMs based on the analysis of the activation distributions in SSMs. We reveal that the distribution of discrete parameters exhibits long-tailed skewness and the distribution of the hidden state sequence exhibits highly dynamic variations. Correspondingly, we design Long-tailed Skewness Quantization (LtSQ) to quantize discrete parameters and Temporal Group Quantization (TGQ) to quantize hidden states, which reduces the quantization errors. Extensive experiments demonstrate that QMamba outperforms advanced PTQ methods on vision models across multiple model sizes and architectures. Notably, QMamba surpasses existing methods by 21.0% on ImageNet classification with 4-bit activations.

Figures

Figures reproduced from arXiv: 2501.13624 by the authors.

**Figure 2.** The statistics of discrete parameters At and hidden states ht are observed in the SSM of Vim [34]. We visualize the distribution range variations of At and ht at different time steps in (a) and (b), where the horizontal axis represents the time dimension. For clarity, we visualize At and ht at every fifth time step in the sequence, and the outliers in the boxplot of hidden states ht are omitted for better visualizat… view at source ↗

**Figure 3.** Analysis of quantization sensitivity in different SSM [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Overview of our QMamba framework. This figure illustrates our quantization framework from the quantized Mamba block to [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: The visualization of our LtSQ and TGQ on discrete parameters [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Theoretical efficiency of the quantized Vim. The size of [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Quantizing Small-Scale State-Space Models for Edge AI
cs.LG 2025-06 conditional novelty 4.0 of 10

Quantization-aware training with a frozen state matrix lifts sequential MNIST accuracy from 40% under post-training quantization to 96%, and a heterogeneous precision scheme cuts memory by 6 times.

Reference graph

Works this paper leans on

36 extracted references · 28 canonical work pages · cited by 1 Pith paper

[1]

PACT: parameterized clipping activation for quantized neural networks

Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. PACT: parameterized clipping activation for quantized neural networks. CoRR, abs/1805.06085,

arXiv
[2]

Low-bit quantization of neural networks for efficient infer- ence

Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient infer- ence. In 2019 IEEE/CVF International Conference on Com- puter Vision Workshops, ICCV Workshops 2019, Seoul, Ko- rea (South), October 27-28, 2019 , pages 3009–3018. IEEE,

work page 2019
[3]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Rep- resentations, ICLR 20...

2021
[4]

Esser, Jeffrey L

Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned step size quantization. In 8th International Confer- ence on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. 2

work page 2020
[5]

Differen- tiable soft quantization: Bridging full-precision and low-bit neural networks

Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differen- tiable soft quantization: Bridging full-precision and low-bit neural networks. In 2019 IEEE/CVF International Confer- ence on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019 , pages 4851–4860. IEEE,

work page 2019
[6]

Mamba: Linear-time sequence mod- eling with selective state spaces

Albert Gu and Tri Dao. Mamba: Linear-time sequence mod- eling with selective state spaces. CoRR, abs/2312.00752,

arXiv
[7]

Combining recurrent, convolutional, and continuous-time models with linear state space layers

Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher R ´e. Combining recurrent, convolutional, and continuous-time models with linear state space layers. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Pro- cessing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, p...

work page 2021
[8]

On the parameterization and initialization of diagonal state space models

Albert Gu, Karan Goel, Ankit Gupta, and Christopher R ´e. On the parameterization and initialization of diagonal state space models. In Advances in Neural Information Process- ing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022

work page 2022

Show all 36 references

[9]

Efficiently modeling long sequences with structured state spaces

Albert Gu, Karan Goel, and Christopher R ´e. Efficiently modeling long sequences with structured state spaces. In The Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 2022. Open- Review.net, 2022. 1, 2

2022
[10]

Mambair: A simple baseline for image restoration with state-space model

Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, and Shu-Tao Xia. Mambair: A simple baseline for image restoration with state-space model. In Computer Vi- sion - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XVIII ,...

2024
[11]

Diagonal state spaces are as effective as structured state spaces

Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as structured state spaces. In Advances in Neural Information Processing Systems 35: An- nual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novem...

2022
[12]

Demystify mamba in vision: A linear attention perspective

Dongchen Han, Ziyi Wang, Zhuofan Xia, Yizeng Han, Yi- fan Pu, Chunjiang Ge, Jun Song, Shiji Song, Bo Zheng, and Gao Huang. Demystify mamba in vision: A linear attention perspective. CoRR, abs/2405.16605, 2024. 4

2024 arXiv
[13]

Howard, Hartwig Adam, and Dmitry Kalenichenko

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neu- ral networks for efficient integer-arithmetic-only inference. In 2018 IEEE Conference on Computer Vision and Pattern...

2018
[14]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. 6

2015
[15]

BRECQ: pushing the limit of post-training quantization by block re- construction

Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. BRECQ: pushing the limit of post-training quantization by block re- construction. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May...

2021
[16]

Repq- vit: Scale reparameterization for post-training quantization of vision transformers

Zhikai Li, Junrui Xiao, Lianwei Yang, and Qingyi Gu. Repq- vit: Scale reparameterization for post-training quantization of vision transformers. In IEEE/CVF International Confer- ence on Computer Vision, ICCV 2023, Paris, France, Octo- ber 1-6, 2023, pages 17181–17190. IEEE, 2023. 1, 5

2023
[17]

Fq-vit: Post-training quantization for fully quantized vision transformer

Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. Fq-vit: Post-training quantization for fully quantized vision transformer. In Proceedings of the Thirty- First International Joint Conference on Artificial Intelli- gence, IJCAI 2022, Vienna, Austria, 23-29 July ...

2022
[18]

Vmamba: Visual state space model

Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. CoRR, abs/2401.10166, 2024. 1, 2, 6, 8

2024 arXiv
[19]

Bi-real net: Enhancing the per- formance of 1-bit cnns with improved representational capa- bility and advanced training algorithm

Zechun Liu, Baoyuan Wu, Wenhan Luo, Xin Yang, Wei Liu, and Kwang-Ting Cheng. Bi-real net: Enhancing the per- formance of 1-bit cnns with improved representational capa- bility and advanced training algorithm. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, G...

2018
[20]

SGDR: stochastic gradient descent with warm restarts

Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenRe- view.net, 2017. 6

2017
[21]

PTQ4SAM: post-training quantization for segment anything

Chengtao Lv, Hong Chen, Jinyang Guo, Jinyang Guo, Jinyang Guo, Yifu Ding, and Xianglong Liu. PTQ4SAM: post-training quantization for segment anything. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 15941–...

2024
[22]

Long range language modeling via gated state spaces

Harsh Mehta, Ankit Gupta, Ashok Cutkosky, and Behnam Neyshabur. Long range language modeling via gated state spaces. In The Eleventh International Conference on Learn- ing Representations, ICLR 2023, Kigali, Rwanda, May 1-5,

2023
[23]

Instance-aware group quantization for vision transformers

Jaehyeon Moon, Dohyung Kim, Junyong Cheon, and Bum- sub Ham. Instance-aware group quantization for vision transformers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 16132–16141. IEEE, 2024. 5

2024
[24]

Up or down? adap- tive rounding for post-training quantization

Markus Nagel, Rana Ali Amjad, Mart van Baalen, Chris- tos Louizos, and Tijmen Blankevoort. Up or down? adap- tive rounding for post-training quantization. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , pages 7197–

2020
[25]

Training data-efficient image transformers & distillation through at- tention

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herv ´e J´egou. Training data-efficient image transformers & distillation through at- tention. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 Ju...

2021
[26]

Tri-plane mamba: Efficiently adapting segment anything model for 3d medical images

Hualiang Wang, Yiqun Lin, Xinpeng Ding, and Xiaomeng Li. Tri-plane mamba: Efficiently adapting segment anything model for 3d medical images. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 - 27th International Conference, Marrakesh, Morocco, October...

2024
[27]

Selective struc- tured state-spaces for long-form video understanding

Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, and Raffay Hamid. Selective struc- tured state-spaces for long-form video understanding. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17- 24, 2023...

2023
[28]

Quantformer: Learning extremely low-precision vision transformers

Ziwei Wang, Changyuan Wang, Xiuwei Xu, Jie Zhou, and Jiwen Lu. Quantformer: Learning extremely low-precision vision transformers. IEEE Trans. Pattern Anal. Mach. Intell., 45(7):8813–8826, 2023. 5

2023
[29]

Qdrop: Randomly dropping quantiza- tion for extremely low-bit post-training quantization

Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qdrop: Randomly dropping quantiza- tion for extremely low-bit post-training quantization. In The Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 2022. Open-...

2022
[30]

Integer quantization for deep learn- ing inference: Principles and empirical evaluation

Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quantization for deep learn- ing inference: Principles and empirical evaluation. CoRR, abs/2004.09602, 2020. 6, 7

2004 arXiv
[31]

Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation

Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu. Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 - 27th International Conference, Marrakesh, Morocco, October ...

2024
[32]

Low-bit quantization needs good dis- tribution

Haibao Yu, Tuopu Wen, Guangliang Cheng, Jiankai Sun, Qi Han, and Jianping Shi. Low-bit quantization needs good dis- tribution. In 2020 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR Workshops 2020, Seat- tle, WA, USA, June 14-19, 2020 , pages 2909–2918....

2020
[33]

Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization

Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XII ...

2022
[34]

Vision mamba: Efficient visual representation learning with bidirectional state space model

Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27...

2024
[2021]

OpenReview.net, 2021. 1

2021
[2023]

OpenReview.net, 2023. 1

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

PACT: parameterized clipping activation for quantized neural networks

Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. PACT: parameterized clipping activation for quantized neural networks. CoRR, abs/1805.06085,

arXiv

[2] [2]

Low-bit quantization of neural networks for efficient infer- ence

Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient infer- ence. In 2019 IEEE/CVF International Conference on Com- puter Vision Workshops, ICCV Workshops 2019, Seoul, Ko- rea (South), October 27-28, 2019 , pages 3009–3018. IEEE,

work page 2019

[3] [3]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Rep- resentations, ICLR 20...

2021

[4] [4]

Esser, Jeffrey L

Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned step size quantization. In 8th International Confer- ence on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. 2

work page 2020

[5] [5]

Differen- tiable soft quantization: Bridging full-precision and low-bit neural networks

Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differen- tiable soft quantization: Bridging full-precision and low-bit neural networks. In 2019 IEEE/CVF International Confer- ence on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019 , pages 4851–4860. IEEE,

work page 2019

[6] [6]

Mamba: Linear-time sequence mod- eling with selective state spaces

Albert Gu and Tri Dao. Mamba: Linear-time sequence mod- eling with selective state spaces. CoRR, abs/2312.00752,

arXiv

[7] [7]

Combining recurrent, convolutional, and continuous-time models with linear state space layers

Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher R ´e. Combining recurrent, convolutional, and continuous-time models with linear state space layers. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Pro- cessing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, p...

work page 2021

[8] [8]

On the parameterization and initialization of diagonal state space models

Albert Gu, Karan Goel, Ankit Gupta, and Christopher R ´e. On the parameterization and initialization of diagonal state space models. In Advances in Neural Information Process- ing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022

work page 2022