Pith. sign in

REVIEW 5 major objections 6 minor 49 references

A probabilistic framework for dynamic quantization

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that per-layer quantization parameters can be estimated before the layer runs from a probabilistic surrogate of pre-activations, making dynamic 8-bit quantization feasible at static-quantization memory cost.

desk verdict A practical input-adaptive quantization scheme with a solid empirical eval, but Eq. (12) as printed is not a variance and needs a code check before trusting the numbers. read the letter →

arxiv 2505.10689 v1 pith:4A6GUOTJ submitted 2025-05-15 cs.LG cs.CV

Gabriele Santini , Francesco Paissan , Elisabetta Farella This is my paper · ORCID
classification cs.LGcs.CV
keywords NeuralnetworkquantizationDynamicProbabilisticsurrogatePre-activationstatisticsMemory-efficientinferenceEmbeddedmachinelearningComputervisionbenchmarks8-bit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dynamic quantization is more accurate than static quantization because it measures each input's true activation range, but that measurement forces the device to store an entire layer output, which small embedded systems often cannot afford. The paper claims that the range can instead be predicted before the layer executes, by treating the layer's weights as independent Gaussians and propagating the input through closed-form mean and variance equations for the pre-activations. A short calibration pass fixes an asymmetric interval around those predicted moments, and that interval supplies the scale and zero-point for standard affine quantization. In 8-bit experiments across five vision tasks, the predicted ranges stay within a small margin of full dynamic quantization while using constant memory, and they degrade less than static quantization under input corruption.

What carries the argument

Pre-activation surrogate: closed-form first and second moments of a layer's output derived from the assumption that weights are i.i.d. normal ($W_{ij} \sim \mathcal{N}(\mu_W, \sigma_W^2)$). For a linear layer, $\mathbb{E}[y_j] = \mu_W \sum_i x_i$ and $\mathrm{Var}[y_j] = \sigma_W^2 \sum_i x_i^2$; for a convolution, the sums run over the kernel window and input channels. A calibration set is used only to fix $\alpha$ and $\beta$, the edges of an asymmetric interval $[\mu_y - \alpha\sigma_y, \mu_y + \beta\sigma_y]$ around the predicted moments; the interval is then treated as the dynamic range in the standard affine quantization formula for scale and zero-point. The sampling stride $\gamma$ subsamples output positions to reduce estimation cost quadratically.

What would settle it

Take a trained model whose weights are deliberately correlated—for example a low-rank or group-convolution network—compute the surrogate's predicted pre-activation interval for a batch of inputs, and compare it with the actual min and max of the output tensor; if the predicted interval is systematically too narrow or off-center by an amount the calibration parameters $\alpha, \beta$ cannot absorb, per-channel accuracy should fall below the static-quantization baseline, which would pin the failure on the Gaussian i.i.d. weight assumption.

Watch

Extended reading notes

Core claim

The paper's central claim is that the costly part of dynamic quantization—buffering an entire output tensor to measure its actual min and max—can be replaced by a prediction. For a linear layer $y = Wx$ whose weights are modeled as i.i.d. Gaussian, the surrogate gives $\mathbb{E}[y_j] = \mu_W \sum_i x_i$ and $\mathrm{Var}[y_j] = \sigma_W^2 \sum_i x_i^2$, and the same reasoning is extended to convolutions. These two numbers are computed per input before the layer executes, then rescaled by calibrated asymmetry parameters $\alpha, \beta$ into a quantization interval. In the authors' 8-bit experiments across detection, segmentation, pose estimation, oriented boxes, and classification, the surrogate-based method is consistently the second-best strategy: average in-domain degradation is $0.88\%$ per-channel (vs. $0.54\%$ for full dynamic quantization) and $1.64\%$ per-tensor (vs. $1.09\%$), while memory overhead is constant instead of growing with output resolution.

Load-bearing premise

A trained network's weights are close enough to independent and identically distributed Gaussians that each layer's pre-activation mean and variance can be predicted from the input alone; the paper concedes that this does not provably hold for trained networks.

Editorial extensions

If this is right

  • Dynamic quantization becomes usable on microcontrollers and other working-memory-limited hardware, since the estimation adds a constant $2b'$-bit overhead instead of an $O(HW)$ output buffer.
  • The quantization latency can be tuned with the sampling stride $\gamma$ after deployment; in the authors' per-channel tests, accuracy stays nearly flat even when the estimate is subsampled by a factor of 1024.
  • Calibration is light: on ImageNet with ResNet50, 16 calibration images were enough to reach near-peak per-channel performance.
  • Under corrupted inputs, the method is more robust than static quantization, with average out-of-domain degradations of $2.58\%$ per-tensor and $0.99\%$ per-channel against $5.62\%$ and $12.6\%$ for static quantization.
  • Because the linear-layer estimation cost is $O(d)$ regardless of output length, the method scales to layers with very large output tensors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The surrogate is cheaper than the layer itself for linear layers because its cost is $O(d)$ and independent of output size; this suggests the same prediction could be reused to pick per-input, per-layer bit-widths, giving input-adaptive mixed precision without extra buffering.
  • The Gaussian ansatz is a plug-in choice. Because the framework only consumes first and second moments of pre-activations, a sparsity-aware or heavy-tailed model of ReLU outputs could replace Eqs. (8)-(11) and potentially close more of the gap to true dynamic quantization.
  • The paper demonstrates the trade-off on one microcontroller; a natural extension is a device sweep that maps latency, memory, and accuracy together to show where the method overtakes static or dynamic quantization.
  • Since the calibration interval is fixed on clean data, an online update of $\alpha, \beta$ from the incoming stream of observed pre-activations is a plausible way to recover the remaining out-of-domain gap to dynamic quantization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The manuscript proposes a probabilistic surrogate model for the pre-activations of linear and convolutional layers, using the input tensor's sum and sum-of-squares to estimate the mean and variance of the output before the layer executes. These estimates define an asymmetric quantization interval I(α,β) whose parameters α and β are tuned on a calibration set, enabling per-input adaptive quantization with a memory overhead similar to static quantization. The method is evaluated on object detection, segmentation, pose estimation, oriented bounding boxes, and image classification, and is implemented in CMSIS-NN on an STM32L476RG board, with code released for PyTorch and embedded C.

Significance. If the method works as claimed, it addresses a real deployment bottleneck: dynamic quantization's O(HW) output-tensor buffering prevents its use on memory-constrained devices. The paper's strengths include a released implementation for PyTorch and embedded C, on-device latency measurements, and a broad task coverage with out-of-domain evaluation. However, the central estimator contains a clear mathematical error (Eq. (12)), and the method's core assumption of i.i.d. Gaussian weights is unvalidated, so the empirical results must be interpreted with caution.

major comments (5)
  1. [Section 4.1, Eq. (12)] The aggregated variance as printed is not a variance: it omits the 1/(HWp) normalization and squares the per-entry variance term. The correct pooling of per-location Gaussian moments is (1/(HWp)) Σ_{v,i,j} [Var[y_ijv] + (E[y_ijv] − E[y])²]. Because Var[y_ijv] in Eq. (11) scales with the local sum of squared inputs, the printed σ_y would scale quadratically with input magnitude, a bias that the fixed α,β cannot absorb. Since the results in Tables 1 and 2 rely on this estimator, the formula must be corrected and the released code must be checked against the corrected version.
  2. [Section 4.1] The derivation of the surrogate rests on the assumption that 'neural network weights are i.i.d. and normally distributed,' which the paper itself concedes does not provably hold for trained networks. No evidence is provided that the trained weights used in the benchmarks satisfy this assumption (e.g., weight histograms, correlation statistics). A concrete test would be to compare the estimated μ_y and σ_y against the actual pre-activation statistics on the evaluation checkpoints; if the surrogate is biased, the fixed α,β cannot fully absorb the bias.
  3. [Tables 1 and 2, Section 6.2] The classification rows mix units: the FP32 column reports top-1 accuracy as a percentage (e.g., 80.858) while all quantized entries are reported as proportions (e.g., 0.7764). This obscures the reported degradations. Moreover, the claim 'Our method is always the second-best' is contradicted by Table 1: for ResNet50 per-tensor, static quantization achieves 0.7797 while the proposed method achieves 0.7764. Please correct the units and the statement.
  4. [Section 4.1, Eq. (13)] The tuning procedure for α and β is not described: the target coverage percentage, the search algorithm, and whether coverage is evaluated on actual pre-activations or on the surrogate's predictions are all unspecified. Since α and β are fixed after calibration and directly control the quantization range, the paper should specify this procedure for reproducibility and for judging how sensitive the results are to this choice.
  5. [Section 6.2, Tables 1 and 2] No error bars or multiple-seed runs are reported for the performance results. Given the sensitivity of quantization to the calibration set (see Fig. 5), the claims of 'negligible loss' and 'best tradeoff' would be considerably strengthened by reporting mean and standard deviation over several independent runs, especially because the reported differences between methods are often within about 0.5%.
minor comments (6)
  1. [Section 4.1, after Eq. (11)] The sentence 'Contrary to what we observed in Eqs. (8) and (11)' should likely refer to Eqs. (8) and (9), since the comparison is with the linear layer's variance formula.
  2. [Figure 3] The color assignments in the caption ('estimation' red, 'convolution' green, 'total' blue) are not consistent with the body text, which describes the overall latency 'in green' and the proposed strategy 'in blue'; please align the caption with the legend.
  3. [Section 5.3, Figure 4] The x-axis of Fig. 4 shows a continuous range up to 20, but the text states γ ∈ {1,4,8,16,32}; make the axis discrete or correct the description to match the actual experimental values.
  4. [Section 4.2] The definition of γ ('0 < γ ≤ max((H,W))') is typographically awkward, and the text calls it 'a fraction of the output resolution' although it acts as a sampling stride; please rephrase to avoid confusion.
  5. [References] Reference [19] is about LLM post-training and does not clearly support the sentence on domain distribution shifts; consider replacing it with a more directly relevant reference on covariate shift or domain adaptation.
  6. [Section 6.2] The average degradation figures are reported as 'mAP50-95' percentages but appear to be percentage-point differences; please clarify whether these are relative drops or absolute differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: per-input surrogate moments come from input and weight statistics, and the alpha,beta calibration is disclosed as fixed after calibration.

full rationale

The claimed prediction chain is self-contained. Section 4.1 derives E[y] and Var[y] from the input tensor and assumed weight moments (Eqs. 8-11), so the per-input dynamic-range estimate is a function of the input, not a re-used target. The interval I(alpha,beta) is tuned on a calibration set via Eq. (13) and the paper explicitly states that alpha,beta remain fixed after calibration; this is standard calibration, not a fitted parameter renamed as a prediction. The paper also discloses the surrogate limitation ('this assumption does not provably hold for trained neural networks') and validates empirically, which is the appropriate use of the assumption. Self-citations [2,32,33] appear only as contextual related work on tinyML and are not load-bearing for the derivation. The printed Eq. (12) variance aggregation appears mathematically incorrect (it squares Var[y] and omits normalization), but that is a correctness concern, not a circular reduction of the method to its inputs. No circular step was found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities. Its derivation rests on one strong distributional assumption about trained weights, plus two interval hyperparameters fitted to calibration data. The variance aggregation formula in Eq. (12) also appears to lack normalization, which is a correctness issue rather than an assumption.

free parameters (3)
  • alpha = not reported
    Tuned on the calibration set via Eq. (13) to cover a target percentage of pre-activations; defines the lower bound of the quantization interval, so it directly sets scale and zero-point.
  • beta = not reported
    Tuned on the calibration set via Eq. (13); defines the upper bound of the quantization interval.
  • sampling stride gamma = 1 in main results; 4 selected for calibration-size ablation; varied {1,4,8,16,32}
    User-chosen hyperparameter that trades estimation cost quadratically against accuracy; not fitted to data but hand-set.
assumptions (3)
  • ad hoc to paper Trained network weights are i.i.d. and normally distributed (Section 4.1: 'neural network weights are i.i.d. and normally distributed').
    Used to derive E[y] and Var[y] for linear and conv layers. The paper explicitly concedes this does not provably hold for trained networks.
  • domain assumption Pre-activation distributions are adequately summarized by a mean, a variance, and a fixed asymmetric interval [mu - alpha sigma, mu + beta sigma] once alpha and beta are calibrated.
    Underpins the quantization parameter extraction in Section 4.1; no evidence is given that the distribution shape is stable across layers, inputs, or domain shifts.
  • domain assumption A 16-image calibration set is representative for fitting alpha and beta across all tasks and out-of-domain corruptions.
    Section 5.2 sets the calibration set to 16 training images; the sensitivity study varies calibration size downstream, but the main tradeoff claim depends on this default transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A probabilistic framework for dynamic quantization." pith.science (2026). https://pith.science/paper/4A6GUOTJ

@misc{pith2026250510689,
  author       = {Pith},
  title        = {Pith review of: A probabilistic framework for dynamic quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4A6GUOTJ}},
  note         = {Machine review of arXiv:2505.10689}
}
read the original abstract

We propose a probabilistic framework for dynamic quantization of neural networks that allows for a computationally efficient input-adaptive rescaling of the quantization parameters. Our framework applies a probabilistic model to the network's pre-activations through a lightweight surrogate, enabling the adaptive adjustment of the quantization parameters on a per-input basis without significant memory overhead. We validate our approach on a set of popular computer vision tasks and models, observing only a negligible loss in performance. Our method strikes the best performance and computational overhead tradeoff compared to standard quantization strategies.

Figures

Figures reproduced from arXiv: 2505.10689 by the authors.

Figure 1
Figure 1. This diagram showcases the differences between static (a), dynamic (b) and our quantization strategies (c) for a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. This illustration showcases the effect of some of the corruptions we implemented to simulate domain shifts in the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. This diagram showcases the latency analysis of convolution. We vary independently the number of input and output [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Impact of sampling stride, 𝛾, on per-tensor (T) and per-channel (C) quantization. on-device dynamically-quantized inference. Compared to static quantization, our approach yields better performance, especially in the per-channel setting, where the mAP difference can be …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 24 canonical work pages

  1. [1]

    Ibrahim M Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. 2022. Revis- iting neural scaling laws in language and vision. Advances in Neural Information Processing Systems 35 (2022), 22300–22312

    work page 2022

  2. [2]

    Alberto Ancilotto, Francesco Paissan, and Elisabetta Farella. 2023. XiNet: Efficient Neural Networks for tinyML.2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2023), 16922–16931. https://api.semanticscholar.org/CorpusID: 267026482

    work page 2023

  3. [3]

    Deepika Bablani, Jeffrey L Mckinstry, Steven K Esser, Rathinakumar Appuswamy, and Dharmendra S Modha. 2023. Efficient and effective methods for mixed precision neural network quantization for faster, energy-efficient inference.arXiv preprint arXiv:2301.13330 (2023)

    arXiv 2023

  4. [4]

    Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma

    work page

  5. [5]

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 (2013)

    arXiv 2013

  6. [6]

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feicht- enhofer, and Judy Hoffman. 2022. Token Merging: Your ViT But Faster. ArXiv abs/2210.09461 (2022). https://api.semanticscholar.org/CorpusID:252968113

    arXiv 2022

  7. [7]

    Tri Dao. [n. d.]. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. In The Twelfth International Conference on Learning Representations

    work page

  8. [8]

    Robert David, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Tiezhen Wang, et al. 2021. Tensorflow lite micro: Embedded machine learning for tinyml systems. Proceedings of Machine Learning and Systems 3 (2021), 800–811

    2021

Show all 49 references
  1. [9]

    Alexandre Défossez, Yossi Adi, and Gabriel Synnaeve. [n. d.]. Differentiable Model Compression via Pseudo Quantization Noise. Transactions on Machine Learning Research ([n. d.])

  2. [10]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Ima- geNet: A large-scale hierarchical image database. In2009 IEEE Conference on Com- puter Vision and Pattern Recognition . 248–255. doi:10.1109/CVPR.2009.5206848

    2009

  3. [11]

    Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2020. Hawq-v2: Hessian aware trace-weighted quantization of neural networks. Advances in neural information processing systems 33 (2020), 18518–18529

    2020

  4. [12]

    Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer

  5. [13]

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. 2020. Learned Step Size Quantization. InInternational Conference on Learning Representations

    2020

  6. [14]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  7. [15]

    Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780

    1997

  8. [16]

    Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2018. Quantized neural networks: Training neural networks with low precision weights and activations. journal of machine learning research 18, 187 Preprint. A probabilistic framework for dynamic...

    2018

  9. [17]

    Glenn Jocher and Jing Qiu. 2024. Ultralytics YOLO11. https://github.com/ ultralytics/ultralytics

    2024

  10. [18]

    Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 (2018)

    2018 arXiv

  11. [19]

    Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Salman Khan, and Fahad Shahbaz Khan. 2025. LLM Post-Training: A Deep Dive into Reasoning Large Language Models. arXiv preprint arXiv:2502.21321 (2025)

    2025 arXiv

  12. [20]

    Liangzhen Lai, Naveen Suda, and Vikas Chandra. 2018. Cmsis-nn: Efficient neural network kernels for arm cortex-m cpus. arXiv preprint arXiv:1801.06601 (2018)

    2018 arXiv

  13. [21]

    Jaehoon Lee, Jascha Sohl-dickstein, Jeffrey Pennington, Roman Novak, Sam Schoenholz, and Yasaman Bahri. 2018. Deep Neural Networks as Gaussian Processes. In International Conference on Learning Representations . https:// openreview.net/forum?id=B1EA-M-0Z

    2018

  14. [22]

    Yunsheng Li, Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Lu Yuan, Zicheng Liu, Lei Zhang, and Nuno Vasconcelos. 2021. Micronet: Improving image recognition with extremely low flops. In Proceedings of the IEEE/CVF International conference on computer vision . 468–477

    2021

  15. [23]

    Ji Lin, Wei-Ming Chen, Yujun Lin, Chuang Gan, Song Han, et al. 2020. Mcunet: Tiny deep learning on iot devices. Advances in neural information processing systems 33 (2020), 11711–11722

    2020

  16. [24]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In European Conference on Computer Vision . https://api. semanticscholar.org/CorpusID:14113767

    2014

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023), 26286–26296. https://api.semanticscholar. org/CorpusID:263672058

    2023

  18. [26]

    Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, and Yixuan Yuan. 2023. Efficientvit: Memory efficient vision transformer with cascaded group attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14420–14430

    2023

  19. [27]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 9992–10002. https://api.semantics...

    2021

  20. [28]

    Zhenhua Liu, Yunhe Wang, Kai Han, Siwei Ma, and Wen Gao. 2022. Instance- aware dynamic neural network quantization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 12434–12443

    2022

  21. [29]

    Zhenhua Liu, Xinfeng Zhang, Shanshe Wang, Siwei Ma, and Wen Gao. 2021. Evolutionary Quantization of Neural Networks with Mixed-Precision. ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2021), 2785–2789. https://api.semanti...

    2021

  22. [30]

    Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. 2021. A White Paper on Neural Network Quantization. ArXiv abs/2106.08295 (2021). https://api.semanticscholar.org/ CorpusID:235435934

    2021 arXiv

  23. [31]

    Miloš Nikolić, Ghouthi Boukli Hacene, Ciaran Bannon, Alberto Delmas Lascorz, Matthieu Courbariaux, Omar Mohamed Awad, Isak Edo Vivancos, Yoshua Bengio, Vincent Gripon, and Andreas Moshovos. 2024. Bitpruning: Learning bitlengths for aggressive and accurate quantization. In 2024...

    2024

  24. [32]

    Francesco Paissan, Alberto Ancilotto, and Elisabetta Farella. 2021. PhiNets: A Scalable Backbone for Low-power AI at the Edge.ACM Transactions on Embedded Computing Systems 21 (2021), 1 – 18. https://api.semanticscholar.org/CorpusID: 238253101

    2021

  25. [33]

    Francesco Paissan, Elisabetta Farella, et al. 2024. tinyCLAP: distilling constrastive language-audio pretrained models. In Proceedings of INTERSPEECH 2024 . 1685– 1689

    2024

  26. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

  27. [35]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. In International Conference on Machine Learning . https://api. semanticscholar.org/CorpusID:252923993

    2022

  28. [36]

    Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer

    Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer

  29. [37]

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition . 4510–4520

    2018

  30. [38]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  31. [39]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  32. [40]

    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 10674–10685

    High-Resolution Image Synthesis with Latent Diffusion Models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 10674–10685. https://api.semanticscholar.org/CorpusID:245335280

    2021

  33. [41]

    Yingchun Wang, Jingcai Guo, Song Guo, and Weizhan Zhang. 2023. Data Quality- aware Mixed-precision Quantization via Hybrid Reinforcement Learning. IEEE transactions on neural networks and learning systems PP (2023). https://api. semanticscholar.org/CorpusID:256697391

    2023

  34. [42]

    Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. 2018. DOTA: A Large-Scale Dataset for Object Detection in Aerial Images. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  35. [43]

    Tjalling J. Ypma. 1995. Historical Development of the Newton–Raphson Method. SIAM Rev. 37, 4 (1995), 531–551. doi:10.1137/1037125 arXiv:https://doi.org/10.1137/1037125

    1995 doi

  36. [44]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Neural Information Processing Systems . https://api.semanticscholar.org/ CorpusID:13756489

    2017

  37. [45]

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. 2024. Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model. ArXiv abs/2401.09417 (2024). https: //api.semanticscholar.org/CorpusID:267028142 Preprint

    2024 arXiv

  38. [48]

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022. CoCa: Contrastive Captioners are Image-Text Foundation Models. Trans. Mach. Learn. Res. 2022 (2022). https://api.semanticscholar.org/ CorpusID:248512473

    2022

  39. [2019]

    In Proceedings of the IEEE/CVF international conference on computer vision

    Hawq: Hessian aware quantization of neural networks with mixed- precision. In Proceedings of the IEEE/CVF international conference on computer vision. 293–302

  40. [2021]

    In International conference on machine learning

    Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748–8763

  41. [2024]

    Proceedings of the National Academy of Sciences 121, 27 (2024), e2311878121

    Explaining neural scaling laws. Proceedings of the National Academy of Sciences 121, 27 (2024), e2311878121

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.