Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vector quantization and lookup tables push neural inference below 1 bit and raise area efficiency up to 146x.

desk verdict A useful extension of LUT-based acceleration with a credible accuracy story; the big efficiency numbers rest on a simulator and scaled baselines, so treat them as upper bounds until validated. read the letter →

arxiv 2501.10658 v1 pith:O3OGAJBI submitted 2025-01-18 cs.AR cs.AIcs.LG

Guoyu Li (1 , 2) , Shengyu Ye (2) , Chunyun Chen (3) , Yang Wang (2) , Fan Yang (2) , Ting Cao (2) , Cheng Liu (1)
show 4 more authors
Mohamed M. Sabry (3) Mao Yang (2) ((1) University of Chinese Academy of Sciences (2) Microsoft Research (3) NTU Singapore)
This is my paper · ORCID
classification cs.ARcs.AIcs.LG
keywords lookup-tableacceleratorvectorquantizationlow-bitdeeplearningLUTBoostapproximatecomputingco-designspaceexplorationneuralnetworkinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to break the 1-bit floor of neural-network quantization by quantizing vectors rather than scalars. In the LUT-DLA framework, activation vectors are clustered into centroids, all centroid-by-weight products are precomputed into lookup tables, and inference becomes a nearest-centroid search plus a table read, so the usual multiply-accumulate units disappear from the datapath. To make this practical, the paper introduces LUTBoost, a multistage conversion that calibrates centroids on a pretrained model and then jointly trains weights and centroids, and a co-design search engine that selects vector length, centroid count, distance metric, and hardware parallelism. The central claim is that the resulting accelerators beat conventional deep learning accelerators by 1.4-7.0x in power efficiency and 1.5-146.1x in area efficiency while keeping accuracy loss at 0.1-3.8% for CNNs and 1.4-3.0% for transformers. If true, sub-1-bit inference becomes a hardware co-design reality rather than a representation limit.

What carries the argument

The identity that carries the 'extreme low-bit' claim is the equivalent bit width $\lceil \log_2 c\rceil / v$, where $v$ is the vector length and $c$ is the number of centroids per subspace; it yields values as low as 0.3 bit in the paper's experiments. The load-bearing mechanism is the LUT operator: instead of computing $\hat{A}W$, the hardware compares incoming sub-vectors against a codebook of centroids, looks up the precomputed product in a partial-sum table, and accumulates the result. Its two structures are Centroid Computation Modules (CCMs), built from distance processing elements that pipeline L1, L2, or Chebyshev comparisons, and In-Memory Matching Modules (IMMs), which hold the lookup tables and accumulation scratchpad. The LUT-Stationary dataflow is what makes the architecture memory-centric without drowning in memory traffic: it traverses the input matrix row-first so that the codebook and lookup table are reused, reducing on-chip memory for the representative GEMM from roughly 2 MB to 17.3 KB in the paper's comparison.

What would settle it

An independent RTL synthesis and cycle-accurate simulation of the paper's three designs on a 28 nm FD-SOI process, counting every lookup-table reload and scratchpad bank conflict under the paper's DDR4 bandwidth assumptions, would settle whether the claimed 1.5-146.1x area-efficiency and 1.4-7.0x power-efficiency gains over conventional accelerators survive a full memory-system accounting.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a neural network's matrix multiplications can be replaced by a lookup operation whose per-inference cost is nearly independent of model size. During conversion, each layer's activations are split into sub-vectors and clustered; each sub-vector is represented by the index of its nearest centroid. Since the weights are fixed at inference time, every centroid-weight product is precomputed and stored in a partial-sum lookup table, so a forward pass consists of finding centroids, fetching table entries, and accumulating them. LUT-DLA is the hardware realization of that pattern: Centroid Computation Modules pipeline distance comparisons, In-Memory Matching Modules store the partial-sum LUTs and accumulate, and a LUT-Stationary dataflow tiles the GEMM so the same codebook and table stay on-chip. LUTBoost then makes conversion practical by freezing the pretrained weights while calibrating centroids, then jointly fine-tuning weights and centroids with straight-through gradient estimation and a reconstruction loss so that non-differentiable L1 and Chebyshev similarities remain trainable. The paper reports that this combination stays within a few percent of full precision across CNN and transformer benchmarks, with the efficiency gains described above.

Load-bearing premise

The whole sub-1-bit claim rests on treating a centroid index as carrying $\lceil \log_2 c\rceil / v$ bits, even though the centroids and lookup-table entries themselves are full-precision values; if that definition is not accepted, the story becomes about replacing multipliers rather than about arithmetic below one bit.

Editorial extensions

If this is right

  • If the measured ratios hold, the same silicon area can host more than an order of magnitude more inferences per second on transformer workloads, at an accuracy cost of one to three percent.
  • Heavy arithmetic moves offline at conversion time: a deployed LUT-based model needs comparators and adders for centroid matching plus SRAM for tables, not multiplier arrays, so the hardware scales with memory rather than compute.
  • The multistage LUTBoost recipe removes the need to train LUT-based models from scratch, which the paper identifies as the reason prior LUT models stayed small; its reported results on BERT and OPT-125M are the direct consequence.
  • The co-design engine gives a practical way to pick a deployment point: smaller $v$ and larger $c$ buy accuracy at the cost of more centroid comparisons, while L1 and Chebyshev distances buy hardware savings at a fraction of a percent accuracy.
  • Because the paper shows low-bit quantization is orthogonal to the LUT conversion (BF16 distance with INT8 tables costs under 1% accuracy), existing integer and floating-point low-bit accelerators can still be combined with LUT-DLA rather than replaced by it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the 'sub-1-bit' framing is best read as a compression-ratio statement. Centroids and table entries are stored at full precision, so the efficiency gain comes from eliminating multipliers, not from arithmetic on sub-1-bit numbers; a head-to-head comparison should use a scalar-quantized accelerator storing activations at the same true precision rather than the same equivalent-bit label
  • The paper's finding that ResNet18 tolerates more aggressive settings than ResNet20 hints at a scaling law the paper does not claim: larger, more redundant models should survive stronger vector quantization. That is testable by running LUTBoost on progressively larger transformers and plotting accuracy drop against parameter count.
  • A full-system accounting of LUT reloads, scratchpad bank conflicts, and the asynchronous CCM-IMM handshake would likely compress the upper end of the reported 146.1x area-efficiency ratio, since the published numbers come from synthesis and analytical models rather than end-to-end RTL simulation or silicon measurement.
  • Because the paper shows BF16/INT8 quantization is orthogonal, an obvious unexplored direction is a fully quantized codebook and lookup table, which would compound the efficiency gains while keeping the accuracy drop bounded.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes LUT-DLA, a hardware-software co-design framework that uses vector quantization to convert DNNs into lookup-table (LUT)-based models, enabling what the authors term 'extreme low-bit' inference below 1 bit per scalar value. The framework includes a parameterized Chisel-based hardware generator with dedicated Centroid Computation Modules (CCMs) and In-Memory Matching Modules (IMMs) connected by asynchronous FIFOs, a new LUT-Stationary dataflow for on-chip LUT reuse, a multistage training algorithm called LUTBoost that first calibrates centroids and then jointly trains weights and centroids with a reconstruction loss, and a co-design space search engine that prunes the (vector length, centroid count, parallelism) design space using analytical models. The authors report end-to-end accuracy on CNNs (ResNet, VGG, LeNet) and transformers (BERT, DistilBERT, OPT-125M), and hardware results from synthesis at 28nm FD-SOI for three generated designs. The headline claims are power and area efficiency improvements of 1.4–7.0× and 1.5–146.1×, respectively, over published accelerators, with modest accuracy drops, and 6.2–12.0× speedups over NVDLA under similar area.

Significance. If the efficiency and accuracy results hold, LUT-DLA is a significant step toward making LUT-based neural network inference practical: it scales to models as large as OPT-125M, introduces a dataflow designed specifically for table lookup, and provides a training algorithm that avoids training from scratch. The paper makes concrete artifacts available in principle (Chisel generators, synthesized designs), and the accuracy experiments cover a wider model range than prior LUT works. However, the significance is tempered by three validation gaps: the performance numbers come from an unvalidated cycle-accurate simulator, the efficiency comparisons mix across published baselines without same-flow re-synthesis, and the 'sub-1-bit' framing depends on a nonstandard metric. These issues affect the central claims and need to be addressed before the results can be taken at face value.

major comments (5)
  1. [§VII-B, §VII-C, Table VIII] The headline efficiency gains rest on a cycle-accurate simulator whose accuracy is not validated against the synthesized RTL described in the same section. Section VII-C states that LUT-DLA performance is computed with the authors' simulator, while Section VII-B reports GOPS for the same designs without confirming that the simulator's cycle counts match RTL simulation or gate-level simulation. The baselines (NVDLA, Gemmini, ELSA, FACT, RRAM-DNN) are taken from published results and scaled to 28nm via the equations in [54], so the comparison mixes tool flows, memory compilers, and operating conditions. The 146.1× area-efficiency ratio is largely driven by RRAM-DNN, an in-memory design with very low GOPS, rather than by a systematic architectural advantage. Please validate the simulator against RTL simulation for at least a few configurations and re-synthesize the open-source baselines (NVDLA, Gemmini) in the same 28nm FD-SOI flow, or at minimum quantify the sensitivity of each efficiency ratio to the scaling assumptions and clearly state which comparisons are apples-to-apples.
  2. [Abstract, §VII-B, Table VIII] The claimed efficiency improvement ranges of 1.4–7.0× in power efficiency and 1.5–146.1× in area efficiency are not directly reproducible from Table VIII. For example, LUT-DLA Design 1 has a power efficiency of 2.1 GOPS/mW, which is lower than NVDLA-Large (2.7 GOPS/mW) and FACT (2.8 GOPS/mW), so the assertion that LUT-DLA improves power efficiency over these architectures is false for that design. The authors need to state explicitly which design is used for each ratio and provide a per-baseline table of ratios, or revise the headline ranges to be consistent with the data.
  3. [Abstract, §VII-A, Table IV] The reported accuracy-drop ranges are inconsistent with the results in Table IV. The abstract states that L2 distance causes drops of 0.1%–3.1%, but the largest L2 drop in Table IV is 2.62% (ResNet20 on CIFAR100), and the text's example of 3.1% for ResNet32 L2 corresponds to a drop of only 2.28% in the table. Similarly, the L1 drop of 3.4% is not matched by the largest L1 drop of 2.92% in Table IV, and no Chebyshev column appears in Table IV despite the 3.8% figure mentioned in the text. Please reconcile the reported ranges, add the Chebyshev results, and ensure the abstract and Section VII-A cite the correct maximum drops.
  4. [§VII-A, Fig. 12] The comparison with PECAN and PQA is not a fair algorithmic comparison because the authors state that the original implementations are not open-sourced, so they 'align parameter settings using our own training algorithm.' This means the baselines are effectively retrained with LUTBoost, and the reported accuracy improvements reflect the training method and hyperparameters, not the original PECAN/PQA algorithms. The authors should either use the original implementations or explicitly relabel the comparison as 'PECAN/PQA model configurations trained with LUTBoost' and temper the claim that LUTBoost outperforms the prior methods.
  5. [Table V, §VII-A, Fig. 1] The claim of 'extreme low-bit quantization' below 1 bit relies on a nonstandard metric. Equivalent bit is defined as log2(c)/v, which counts the index bits per vector element, not the arithmetic precision of the stored centroids, lookup-table entries, or input activations. For instance, with v=9 and c=8 the equivalent bit is 1/3, yet the hardware still stores centroid vectors and precomputed LUT results at full or high precision. The efficiency savings come from replacing multipliers with table lookups, not from processing data at sub-1-bit arithmetic precision. The authors should compare against real low-bit methods (e.g., binary/ternary or 2-bit quantization) using a physical metric such as bits per activation value, and either justify the 'sub-1-bit' language or revise it to describe index compression rather than arithmetic bit width.
minor comments (6)
  1. [Fig. 1 caption] The caption shows 'equivalent bit-width = V /log2C', which is inverted relative to the definition in Table V (log2(c)/v); please correct the direction of the formula.
  2. [References] References [39] and [40] are the same paper (Micikevicius et al., FP8 formats); please deduplicate.
  3. [§IV-A] The sentence 'The hardware architecture of flexibly decouples the CCM and IMM designs' is grammatically incorrect; please rephrase.
  4. [Table IX] The 'Cycles (k)' column header and footnote describe the entries as 'the proportion of effective MAC cycle in the overall cycle,' but the values appear to be raw cycle counts; please clarify the units and the description.
  5. [§VI-B, Eq. (2)] The memory access model in Eq. (2) is difficult to parse because of missing parentheses and undefined variables (e.g., bit_lut, bit_out); please rewrite with explicit notation and define all symbols.
  6. [General] The paper does not include a code/data availability statement; releasing the simulator, Chisel generators, and training scripts would greatly support reproducibility and strengthen the work.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: hardware efficiency claims rest on independent synthesis and cycle simulation; only the 'equivalent bit-width' framing is self-definitional.

  1. self definitional [Section VII-A, Table V and accompanying text ('Bitwidth and Similarity Evaluation')]
    "During inference, a subvector of the input matrix is represented by the centroid index within its subspace; the equivalent bit can be computed as ⌈log2c⌉/v."

    The paper's 'extreme low-bit' claim is carried by this definitional formula. Equivalent bit width is not measured or derived from hardware; it is simply log2(c)/v, the standard vector-quantization rate expressed as bits per element, relabeled as an 'equivalent bit'. Consequently the assertion that LUT-DLA reaches sub-1-bit operation (e.g., 0.3 bit in Table V) is true by construction of the metric rather than by a physical scalar bit-width reduction. This framing does not feed into the power/area efficiency measurements, which come from synthesis, so it is a framing circularity rather than a forced prediction.

full rationale

The paper's central efficiency claims are not circular: LUT-DLA PPA numbers in Table VIII come from Chisel RTL synthesized with Cadence Genus on 28nm FD-SOI using ARM memory compilers, and performance is measured with a cycle-accurate simulator plus end-to-end inference cycle counts. These values are independent of the accuracy tables and of the 'equivalent bit' definition. The comparisons against NVDLA, Gemmini, and other accelerators use published numbers scaled with generic scaling equations, which raises a fairness and validation risk (especially since the simulator is not validated against the synthesized RTL), but that is a correctness concern, not a circularity. Self-citations to LUT-NN [55] and PIM-DL [32] are used as background and motivation, not as load-bearing evidence for the new architecture, training algorithm, or PPA results; no uniqueness theorem or externally imported conclusion forces the design. The only self-definitional element is the 'equivalent bit-width' metric, which is defined by the paper and then used to frame VQ as 'extreme low-bit quantization'. That metric does not determine the hardware measurements, so the overall circularity score is low.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. Free parameters such as vector length, centroid count, and hardware parallelism are tuned per model via the search engine and are central to the reported accuracy and efficiency. The key unstated assumptions are that vector quantization preserves accuracy enough for practical deployment and that the simulated hardware numbers are representative of real silicon.

free parameters (5)
  • v (vector length) = 3 to 16 (searched)
    Determines the compression ratio and accuracy; selected via co-design search engine.
  • c (number of centroids) = 8 to 64 (searched)
    Controls codebook size and accuracy; selected via co-design search engine.
  • Reconstruction loss penalty = 0.05 for ResNet, 0.01-0.1 for transformers
    Hand-tuned in LUTBoost to balance centroid fidelity and training stability.
  • Learning rates = 1e-3 and 5e-4 for CNNs, 1e-3 and 5e-5 for transformers
    Chosen per training stage and model; not derived from theory.
  • nCCU, nIMM (hardware parallelism) = e.g., nIMM=8, nCCU=2 for one design
    Determined by the co-design search engine to meet area/power constraints.
assumptions (5)
  • domain assumption Neural network activations exhibit semantic similarity, so vector quantization can approximate matrix multiplication with acceptable error.
    This is the foundation of the LUT-based approach; stated in Section II-B and used throughout.
  • standard math K-means clustering converges to centroids that, after training, represent the input activation distribution well.
    Used in the training pipeline; not formally proven in the paper.
  • domain assumption The Straight-Through Estimator provides gradients that are sufficient for training non-differentiable distance metrics.
    Invoked in Section V to train with L1 and Chebyshev distances.
  • domain assumption Hardware synthesis results and cycle-accurate simulations accurately reflect real chip performance.
    All PPA numbers are from Cadence Genus and a custom simulator, not fabricated silicon.
  • domain assumption Process scaling equations from Stillmaker and Baas (2017) correctly normalize area and power across different technology nodes.
    Used in Table VIII to compare with accelerators fabricated at other nodes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator." pith.science (2026). https://pith.science/paper/O3OGAJBI

@misc{pith2026250110658,
  author       = {Pith},
  title        = {Pith review of: LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3OGAJBI}},
  note         = {Machine review of arXiv:2501.10658}
}
abstract

The emergence of neural network capabilities invariably leads to a significant surge in computational demands due to expanding model sizes and increased computational complexity. To reduce model size and lower inference costs, recent research has focused on simplifying models and designing hardware accelerators using low-bit quantization. However, due to numerical representation limits, scalar quantization cannot reduce bit width lower than 1-bit, diminishing its benefits. To break through these limitations, we introduce LUT-DLA, a Look-Up Table (LUT) Deep Learning Accelerator Framework that utilizes vector quantization to convert neural network models into LUTs, achieving extreme low-bit quantization. The LUT-DLA framework facilitates efficient and cost-effective hardware accelerator designs and supports the LUTBoost algorithm, which helps to transform various DNN models into LUT-based models via multistage training, drastically cutting both computational and hardware overhead. Additionally, through co-design space exploration, LUT-DLA assesses the impact of various model and hardware parameters to fine-tune hardware configurations for different application scenarios, optimizing performance and efficiency. Our comprehensive experiments show that LUT-DLA achieves improvements in power efficiency and area efficiency with gains of $1.4$~$7.0\times$ and $1.5$~$146.1\times$, respectively, while maintaining only a modest accuracy drop. For CNNs, accuracy decreases by $0.1\%$~$3.1\%$ using the $L_2$ distance similarity, $0.1\%$~$3.4\%$ with the $L_1$ distance similarity, and $0.1\%$~$3.8\%$ when employing the Chebyshev distance similarity. For transformer-based models, the accuracy drop ranges from $1.4\%$ to $3.0\%$.

Figures

Figures reproduced from arXiv: 2501.10658 by the authors.

Figure 1
Figure 1. Comparison of Area and Power Efficiency: LUT [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. VQ for Approximating Matrix Multiplication [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. illustrates the architecture of the framework, which consists of: LUT-DLA Hardware Generator: To address Challenge 1, we design LUT-DLA Hardware Generator, an agile and parameterized hardware architecture generation framework implemented in Chisel [3]. The hardware architecture gener￾ated by the LUT-DLA Hardware Generator primarily consists of Centroid Calculation Modules (CCMs) and In-Memory Matching Modules (IMMs)… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: LUT-DLA Hardware Architecture dPE1 dPE3 dPE4 Input Vector Min Index Min Dist. & Idx. (From upstream) Distance Compute Reg < Min Dist. & Idx. (New) MSB=0 Euclidean (L2) Manhattan (L1) Chebyshev x x > Reg … - - MSB=0 + Reg - - MSB=0 MSB=0 - - Reg + dPE𝐾𝐾𝑚𝑚𝑚𝑚𝑚𝑚 𝑽𝑽𝒎𝒎𝒎𝒎𝒎𝒎 …
Figure 7
Figure 7. Figure 7: shows the progress of BERT training using LUT￾Boost. Compared to single-stage training [32] (BERT-base, vector=4, centroid=64), the multistage method significantly decreases training loss by centroids in 2000 iterations and achieves faster and better convergence by tra…
Figure 6
Figure 6. Figure 6: LUTBoost: Lightweight Multistage Model Converter [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 9
Figure 9. Figure 9: Left: Area and Energy Overhead of dPE when v = 8. Right: Hardware Overhead Under Different v 2) Hardware Resource Sensitivity: The main factors af￾fecting the computational component are vector length, simi￾larity metrics, and numeric precision. Vector length: Vector l…
Figure 10
Figure 10. Figure 10: Expanding the Lookup-Limited Design to Increase [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Co-Design Space Search Engine Example Algorithm 2 represents our heuristic search [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: Comparison with PECAN and PQA TABLE VII: IMM Settings and Resources Need V Nc Tn M SRAM Bandwidth Design1 3 16 128 256 36.1KB 4.1GB/s Design2 4 16 256 256 72.1KB 7.0GB/s Design3 3 16 768 512 408.2KB 8.7GB/s the equivalent bit width, thus facilitating feature extractio…
Figure 13
Figure 13. Figure 13: End-to-End Throughput and Energy Consumption [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: PPA Analysis following experiments, NVDLA performance is calculated by the official performance model [44], Gemmini cycle count [15] is simulated by Verilator. The performance of LUT-DLA is based on our cycle-accurate simulator, which assumes a maximum off-chip bandwi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness

    cs.AR 2025-09 conditional novelty 7.0 of 10

    A bit-slice-based accelerator (MCBP) jointly reduces GEMM computation, weight traffic, and KV cache traffic for LLM inference, claiming 9.43x speedup and 31.1x energy efficiency over A100.

Reference graph

Works this paper leans on

61 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [54]

    Scaling equations for the accurate prediction of CMOS device performance from 180 nm to 7 nm,

    A. Stillmaker and B. M. Baas, “Scaling equations for the accurate prediction of CMOS device performance from 180 nm to 7 nm,” Integr., vol. 58, pp. 74–81, 2017. [Online]. Available: https: //doi.org/10.1016/j.vlsi.2017.02.002

    doi:10.1016/j.vlsi.2017.02.002 2017

  2. [1]

    Pqa: Exploring the potential of product quantization in dnn hardware acceleration,

    A. F. AbouElhamayed, A. Cui, J. Fernandez-Marques, N. D. Lane, and M. S. Abdelfattah, “Pqa: Exploring the potential of product quantization in dnn hardware acceleration,” ACM Trans. Reconfigurable Technol. Syst. , apr 2024, just Accepted. [Online]. Available: https://doi.org/10.1145/3656643

    work page doi:10.1145/3656643 2024

  3. [2]

    Hardware approximate techniques for deep neural network accelerators: A survey,

    G. Armeniakos, G. Zervakis, D. Soudris, and J. Henkel, “Hardware approximate techniques for deep neural network accelerators: A survey,” ACM Comput. Surv. , vol. 55, no. 4, nov 2022. [Online]. Available: https://doi.org/10.1145/3527156

    doi:10.1145/3527156 2022

  4. [3]

    Chisel: constructing hardware in a scala embedded language,

    J. Bachrach, H. V o, B. Richards, Y . Lee, A. Waterman, R. Avi ˇzienis, J. Wawrzynek, and K. Asanovi ´c, “Chisel: constructing hardware in a scala embedded language,” in DAC Design automation conference 2012. IEEE, 2012, pp. 1212–1221

    work page 2012

  5. [4]

    Multiplying matrices without multiplying,

    D. W. Blalock and J. V . Guttag, “Multiplying matrices without multiplying,” in Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 2021, pp. 992–1004. [Online]. Available: http://proceedings.mlr.press/v...

    work page 2021

  6. [5]

    RTX on - the NVIDIA turing GPU,

    J. Burgess, “RTX on - the NVIDIA turing GPU,” IEEE Micro , vol. 40, no. 2, pp. 36–44, 2020. [Online]. Available: https: //doi.org/10.1109/MM.2020.2971677

    arXiv 2020

  7. [6]

    Deepburning-seg: Generating DNN accelerators of segment-grained pipeline architecture,

    X. Cai, Y . Wang, X. Ma, Y . Han, and L. Zhang, “Deepburning-seg: Generating DNN accelerators of segment-grained pipeline architecture,” in 55th IEEE/ACM International Symposium on Microarchitecture, MICRO 2022, Chicago, IL, USA, October 1-5, 2022 . IEEE, 2022, pp. 1396–1413. [Online]. Available: https://doi.org/10.1109/MICRO56248. 2022.00094

    work page arXiv 2022

  8. [7]

    QuIP: 2-bit quantization of large language models with guarantees,

    J. Chee, Y . Cai, V . Kuleshov, and C. D. Sa, “QuIP: 2-bit quantization of large language models with guarantees,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=xrk9g5vcXR

    work page 2023

Show all 61 references
  1. [8]

    NVIDIA hopper H100 GPU: scaling performance,

    J. Choquette, “NVIDIA hopper H100 GPU: scaling performance,” IEEE Micro , vol. 43, no. 3, pp. 9–17, 2023. [Online]. Available: https://doi.org/10.1109/MM.2023.3256796

    2023

  2. [9]

    NVIDIA A100 tensor core GPU: performance and innovation,

    J. Choquette, W. Gandhi, O. Giroux, N. Stam, and R. Krashinsky, “NVIDIA A100 tensor core GPU: performance and innovation,” IEEE Micro , vol. 41, no. 2, pp. 29–35, 2021. [Online]. Available: https://doi.org/10.1109/MM.2021.3061394

    2021

  3. [10]

    Using vector quantization for image processing,

    P. Cosman, K. Oehler, E. Riskin, and R. Gray, “Using vector quantization for image processing,” Proceedings of the IEEE , vol. 81, no. 9, pp. 1326–1341, 1993

    1993

  4. [11]

    GPTQ: accurate post-training quantization for generative pre-trained transformers,

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “GPTQ: accurate post-training quantization for generative pre-trained transformers,” CoRR, vol. abs/2210.17323, 2022. [Online]. Available: https://doi.org/ 10.48550/arXiv.2210.17323

    2022 doi arXiv

  5. [12]

    The accelerator wall: Limits of chip specialization,

    A. Fuchs and D. Wentzlaff, “The accelerator wall: Limits of chip specialization,” in 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019, Washington, DC, USA, February 16-20, 2019 . IEEE, 2019, pp. 1–14. [Online]. Available: https://doi.org/1...

    2019

  6. [13]

    Optimized product quantization for approximate nearest neighbor search,

    T. Ge, K. He, Q. Ke, and J. Sun, “Optimized product quantization for approximate nearest neighbor search,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, June 23-28, 2013. IEEE Computer Society, 2013, pp. 2946–2953. [Online]. Available: ...

    2013 doi

  7. [14]

    Optimized product quantization,

    T. Ge, K. He, Q. Ke, and J. Sun, “Optimized product quantization,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 36, no. 4, pp. 744–755, 2014

    2014

  8. [15]

    Gemmini: Enabling systematic deep- learning architecture evaluation via full-stack integration,

    H. Genc, S. Kim, A. Amid, A. Haj-Ali, V . Iyer, P. Prakash, J. Zhao, D. Grubb, H. Liew, H. Mao et al., “Gemmini: Enabling systematic deep- learning architecture evaluation via full-stack integration,” in 2021 58th ACM/IEEE Design Automation Conference (DAC) . IEEE, 2021, pp. 769–774

    2021

  9. [16]

    Vector quantization,

    R. Gray, “Vector quantization,” IEEE ASSP Magazine, vol. 1, no. 2, pp. 4–29, 1984

    1984

  10. [18]

    Ant: Exploiting adaptive numerical data type for low-bit deep neural network quantization,

    C. Guo, C. Zhang, J. Leng, Z. Liu, F. Yang, Y . Liu, M. Guo, and Y . Zhu, “Ant: Exploiting adaptive numerical data type for low-bit deep neural network quantization,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2022, pp. 1414–1433

    2022

  11. [19]

    NNPIM: A processing in-memory architecture for neural network acceleration,

    S. Gupta, M. Imani, H. Kaur, and T. S. Rosing, “NNPIM: A processing in-memory architecture for neural network acceleration,” IEEE Trans. Computers, vol. 68, no. 9, pp. 1325–1337, 2019. [Online]. Available: https://doi.org/10.1109/TC.2019.2903055

    2019

  12. [20]

    ELSA: hardware-software co-design for efficient, lightweight self- attention mechanism in neural networks,

    T. J. Ham, Y . Lee, S. H. Seo, S. Kim, H. Choi, S. J. Jung, and J. W. Lee, “ELSA: hardware-software co-design for efficient, lightweight self- attention mechanism in neural networks,” in 48th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2021, Virtual ...

    2021

  13. [21]

    Approximate computing: An emerging paradigm for energy-efficient design,

    J. Han and M. Orshansky, “Approximate computing: An emerging paradigm for energy-efficient design,” in 2013 18th IEEE European Test Symposium (ETS). IEEE, 2013, pp. 1–6

    2013

  14. [22]

    EIE: efficient inference engine on compressed deep neural network,

    S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “EIE: efficient inference engine on compressed deep neural network,” in 43rd ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2016, Seoul, South Korea, June 18-22, 2016 . IEEE Comp...

    2016 doi

  15. [23]

    Limits to the energy efficiency of cmos microprocessors,

    A. Ho, E. Erdil, and T. Besiroglu, “Limits to the energy efficiency of cmos microprocessors,” in 2023 IEEE International Conference on Rebooting Computing (ICRC) . IEEE, 2023, pp. 1–10

    2023

  16. [24]

    Training compute-optimal large language models,

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifr...

    2022 doi arXiv

  17. [25]

    RAPIDNN: in-memory deep neural network acceleration framework,

    M. Imani, M. Samragh, Y . Kim, S. Gupta, F. Koushanfar, and T. Rosing, “RAPIDNN: in-memory deep neural network acceleration framework,” CoRR, vol. abs/1806.05794, 2018. [Online]. Available: http://arxiv.org/abs/1806.05794

    2018 arXiv

  18. [26]

    Transpimlib: A library for efficient transcendental functions on processing-in-memory systems,

    M. Item, J. G ´omez-Luna, Y . Guo, G. F. Oliveira, M. Sadrosadati, and O. Mutlu, “Transpimlib: A library for efficient transcendental functions on processing-in-memory systems,” CoRR, vol. abs/2304.01951, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2304.01951

    2023 doi arXiv

  19. [27]

    TPU v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings,

    N. P. Jouppi, G. Kurian, S. Li, P. C. Ma, R. Nagarajan, L. Nai, N. Patil, S. Subramanian, A. Swing, B. Towles, C. Young, X. Zhou, Z. Zhou, and D. A. Patterson, “TPU v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings,” in Pro...

    2023

  20. [28]

    In-datacenter performance analysis of a tensor processing unit,

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers et al. , “In-datacenter performance analysis of a tensor processing unit,” in Proceedings of the 44th annual international symposium on computer architecture , 2017...

    2017

  21. [29]

    Product quantization for nearest neighbor search,

    H. J ´egou, M. Douze, and C. Schmid, “Product quantization for nearest neighbor search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 1, pp. 117–128, 2011

    2011

  22. [30]

    Scaling laws for neural language models,

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” CoRR, vol. abs/2001.08361, 2020. [Online]. Available: https://arxiv.org/abs/2001.08361

    2001 arXiv

  23. [31]

    Irreversibility and heat generation in the computing process,

    R. Landauer, “Irreversibility and heat generation in the computing process,” IBM J. Res. Dev. , vol. 5, no. 3, pp. 183–191, 1961. [Online]. Available: https://doi.org/10.1147/rd.53.0183

    1961 doi

  24. [32]

    Pim-dl: Expanding the applicability of commodity dram-pims for deep learning via algorithm-system co-optimization,

    C. Li, Z. Zhou, Y . Wang, F. Yang, T. Cao, M. Yang, Y . Liang, and G. Sun, “Pim-dl: Expanding the applicability of commodity dram-pims for deep learning via algorithm-system co-optimization,” in ASPLOS ’24: 29th ACM International Conference on Architectural Support for Program...

    2024

  25. [33]

    Boosting mobile CNN inference through semantic memory,

    Y . Li, C. Zhang, S. Han, L. L. Zhang, B. Yin, Y . Liu, and M. Xu, “Boosting mobile CNN inference through semantic memory,” in MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 13 - 24, 2021 , H. T. Shen, Y . Zhuang, J. R. Smith, Y . Yang, P. C ´esar, F. Metz...

    2021

  26. [34]

    RRAM-DNN: an RRAM and model-compression empowered all-weights-on-chip DNN accelerator,

    Z. Li, Z. Wang, L. Xu, Q. Dong, B. Liu, C. Su, W. Chu, G. Tsou, Y . Chih, T. J. Chang, D. Sylvester, H. Kim, and D. T. Blaauw, “RRAM-DNN: an RRAM and model-compression empowered all-weights-on-chip DNN accelerator,” IEEE J. Solid State Circuits, vol. 56, no. 4, pp. 1105–1115, ...

    2021

  27. [35]

    AWQ: activation-aware weight quantization for LLM compression and acceleration,

    J. Lin, J. Tang, H. Tang, S. Yang, X. Dang, and S. Han, “AWQ: activation-aware weight quantization for LLM compression and acceleration,” CoRR, vol. abs/2306.00978, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2306.00978

    2023 doi arXiv

  28. [36]

    LLM-FP4: 4-bit floating-point quantized transformers,

    S. Liu, Z. Liu, X. Huang, P. Dong, and K. Cheng, “LLM-FP4: 4-bit floating-point quantized transformers,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 , H. Bouamor, J. Pino, and K. Bali, Ed...

    2023

  29. [37]

    The era of 1-bit llms: All large language models are in 1.58 bits,

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei, “The era of 1-bit llms: All large language models are in 1.58 bits,” CoRR, vol. abs/2402.17764, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2402.17764

    2024 doi arXiv

  30. [38]

    Vector quantization in speech coding,

    J. Makhoul, S. Roucos, and H. Gish, “Vector quantization in speech coding,” Proceedings of the IEEE, vol. 73, no. 11, pp. 1551–1588, 1985

    1985

  31. [40]

    FP8 formats for deep learning,

    P. Micikevicius, D. Stosic, N. Burgess, M. Cornea, P. Dubey, R. Grisenthwaite, S. Ha, A. Heinecke, P. Judd, J. Kamalu, N. Mellempudi, S. F. Oberman, M. Shoeybi, M. Y . Siu, and H. Wu, “FP8 formats for deep learning,” CoRR, vol. abs/2209.05433, 2022. [Online]. Available: https:...

    2022 doi arXiv

  32. [41]

    Energy-efficient convolutional neural networks via recurrent data reuse,

    L. Mocerino, V . Tenace, and A. Calimera, “Energy-efficient convolutional neural networks via recurrent data reuse,” in Design, Automation Test in Europe Conference Exhibition, DATE 2019, Florence, Italy, March 25-29, 2019 , J. Teich and F. Fummi, Eds. IEEE, 2019, pp. 848–853....

    2019

  33. [42]

    Evoapprox8b: Library of approximate adders and multipliers for circuit design and benchmarking of approximation methods,

    V . Mrazek, R. Hrbacek, Z. Vasicek, and L. Sekanina, “Evoapprox8b: Library of approximate adders and multipliers for circuit design and benchmarking of approximation methods,” in Design, Automation Test in Europe Conference Exhibition (DATE), 2017 , 2017, pp. 258–261

    2017

  34. [43]

    Memory-centric computing,

    O. Mutlu, “Memory-centric computing,” CoRR, vol. abs/2305.20000,

    arXiv

  35. [44]

    Nvdla open source hardware performance

    Nvidia. Nvdla open source hardware performance. Accessed on 2023- 11-28. [Online]. Available: https://github.com/nvdla/hw/blob/nvdlav1/ perf

    2023

  36. [45]

    Nvidia deep learning accelerator

    NVIDIA. Nvidia deep learning accelerator. Accessed on 2023-11-13. [Online]. Available: http://nvdla.org

    2023

  37. [46]

    (2024) Nvidia dgx b200 datasheet

    NVIDIA. (2024) Nvidia dgx b200 datasheet. [Online]. Available: https://resources.nvidia.com/en-us-dgx-systems/dgx-b200-datasheet

    2024

  38. [47]

    SCNN: an accelerator for compressed-sparse convolutional neural networks,

    A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. S. Emer, S. W. Keckler, and W. J. Dally, “SCNN: an accelerator for compressed-sparse convolutional neural networks,” in Proceedings of the 44th Annual International Symposium on Computer Architecture...

    2017

  39. [48]

    LUT-GEMM: quantized matrix multiplication based on luts for efficient inference in large- scale generative language models,

    G. Park, B. Park, M. Kim, S. Lee, J. Kim, B. Kwon, S. J. Kwon, B. Kim, Y . Lee, and D. Lee, “LUT-GEMM: quantized matrix multiplication based on luts for efficient inference in large- scale generative language models,” in The Twelfth International Conference on Learning Represe...

    2024

  40. [49]

    FACT: ffn-attention co-optimized transformer architecture with eager correlation prediction,

    Y . Qin, Y . Wang, D. Deng, Z. Zhao, X. Yang, L. Liu, S. Wei, Y . Hu, and S. Yin, “FACT: ffn-attention co-optimized transformer architecture with eager correlation prediction,” in Proceedings of the 50th Annual International Symposium on Computer Architecture, ISCA 2023, Orlan...

    2023

  41. [50]

    PECAN: A product-quantized content addressable memory network,

    J. Ran, R. Lin, J. C. L. Li, J. Zhou, and N. Wong, “PECAN: A product-quantized content addressable memory network,” in Design, Automation Test in Europe Conference Exhibition, DATE 2023, Antwerp, Belgium, April 17-19, 2023 . IEEE, 2023, pp. 1–6. [Online]. Available: https://do...

    2023

  42. [51]

    Computation reuse in dnns by exploiting input similarity,

    M. Riera, J. Arnau, and A. Gonz ´alez, “Computation reuse in dnns by exploiting input similarity,” in 45th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2018, Los Angeles, CA, USA, June 1-6, 2018 , M. Annavaram, T. M. Pinkston, and B. Falsafi, Eds. IEE...

    2018

  43. [52]

    Stella nera: Achieving 161 top/s/w with multiplier-free DNN acceleration based on approximate matrix multiplication,

    J. Sch ¨onleber, L. Cavigelli, R. Andri, M. Perotti, and L. Benini, “Stella nera: Achieving 161 top/s/w with multiplier-free DNN acceleration based on approximate matrix multiplication,” CoRR, vol. abs/2311.10207, 2023. [Online]. Available: https://doi.org/10.48550/ arXiv.2311.10207

    2023 doi arXiv

  44. [53]

    Softermax: Hardware/software co-design of an efficient softmax for transformers,

    J. R. Stevens, R. Venkatesan, S. Dai, B. Khailany, and A. Raghunathan, “Softermax: Hardware/software co-design of an efficient softmax for transformers,” in 58th ACM/IEEE Design Automation Conference, DAC 2021, San Francisco, CA, USA, December 5-9, 2021 . IEEE, 2021, pp. 469–4...

    2021

  45. [55]

    LUT-NN: empower efficient neural network inference with centroid learning and table lookup,

    X. Tang, Y . Wang, T. Cao, L. L. Zhang, Q. Chen, D. Cai, Y . Liu, and M. Yang, “LUT-NN: empower efficient neural network inference with centroid learning and table lookup,” in Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, ACM MobiC...

    2023

  46. [56]

    Weight-oriented approximation for energy-efficient neural network inference accelerators,

    Z.-G. Tasoulas, G. Zervakis, I. Anagnostopoulos, H. Amrouch, and J. Henkel, “Weight-oriented approximation for energy-efficient neural network inference accelerators,” IEEE Transactions on Circuits and Systems I: Regular Papers , vol. 67, no. 12, pp. 4670–4683, 2020

    2020

  47. [57]

    Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks,

    A. Tseng, J. Chee, Q. Sun, V . Kuleshov, and C. D. Sa, “Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks,” 2024

    2024

  48. [58]

    GLUE: A multi-task benchmark and analysis platform for natural language understanding,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman, “GLUE: A multi-task benchmark and analysis platform for natural language understanding,” in Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP , T. Linzen, G. Ch...

    2018

  49. [59]

    Bitnet: Scaling 1-bit transformers for large language models,

    H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y . Wu, and F. Wei, “Bitnet: Scaling 1-bit transformers for large language models,” CoRR, vol. abs/2310.11453, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2310.11453

    2023 doi arXiv

  50. [60]

    Learnable lookup table for neural network quantization,

    L. Wang, X. Dong, Y . Wang, L. Liu, W. An, and Y . Guo, “Learnable lookup table for neural network quantization,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 . IEEE, 2022, pp. 12 413–12 423. [Online]. Ava...

    2022

  51. [61]

    Nn-lut: Neural approximation of non-linear operations for efficient transformer inference,

    J. Yu, J. Park, S. Park, M. Kim, S. Lee, D. H. Lee, and J. Choi, “Nn-lut: Neural approximation of non-linear operations for efficient transformer inference,” in Proceedings of the 59th ACM/IEEE Design Automation Conference , ser. DAC ’22. New York, NY , USA: Association for Co...

    2022

  52. [62]

    Dnnbuilder: an automated tool for building high-performance DNN hardware accelerators for fpgas,

    X. Zhang, J. Wang, C. Zhu, Y . Lin, J. Xiong, W. W. Hwu, and D. Chen, “Dnnbuilder: an automated tool for building high-performance DNN hardware accelerators for fpgas,” in Proceedings of the International Conference on Computer-Aided Design, ICCAD 2018, San Diego, CA, USA, Nov...

    2018

  53. [2023]

    Available: https://doi.org/10.48550/arXiv.2305.20000

    [Online]. Available: https://doi.org/10.48550/arXiv.2305.20000

    doi arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.