Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers

Ching-Yi Lin; Sahil Shah

REVIEW 5 major objections 6 minor 44 references

Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A systolic-array accelerator specialized for 3-bit integerized vision transformers can run multi-head self-attention at 13,568 GOPs/s and 219.4 GOPs/W on a 16nm FPGA, beating same-technology GPUs on throughput and power efficiency.

desk verdict The architecture is genuinely interesting, but the headline throughput number collapses under the paper's own timing equations, so the central performance claim is not currently supported. read the letter →

arxiv 2508.20334 v1 pith:5KS527MC submitted 2025-08-28 eess.SY cs.SY

Ching-Yi Lin , Sahil Shah This is my paper · ORCID

classification eess.SYcs.SY

keywords VisionTransformerFPGAAcceleratorModelIntegerizationQuantizationSystolicArrayHardware-SoftwareCo-designLow-bitArithmeticMulti-headSelf-Attention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a model-specialized accelerator built for a single 3-bit integerized vision transformer can beat general-purpose GPUs on throughput and power efficiency for inference serving. Its target is the multi-head self-attention (MSA) module of DeiT-S, reorganized so that every compute-heavy matrix product runs on 3-bit integers with no dequantized operands, while the integerized model keeps 77.81% top-1 accuracy on ImageNet and 96.83% on CIFAR-10. On a 16 nm Alveo U250 FPGA the synthesis-validated design reports 13,568 GOPs/s at 61.83 W, i.e. 219.4 GOPs/s/W, numbers the authors compare favorably with same-technology GPUs and prior FPGA transformer accelerators. The enabling insight is operational intensity: choosing an offload cut point with high data reuse (4342 OP/byte) and moving only low-bit data between modules lets the accelerator run at 400 MHz with just 3.13 GB/s of bandwidth. If the design works as reported, it points to a serving model in which cheap, low-bandwidth, power-sipping hardware takes over the high-reuse parts of transformer inference.

What carries the argument

The carrying mechanism is a fully systolic array template: a 2D weight-stationary MAC array of 3-bit multiply-accumulate PEs, followed by a post-MAC array, a systolic aggregation module, a triangular delay unit, and a post-aggregation array, with a shift-register-and-latch weight-loading unit for dynamically generated weights. This template maps every MSA operation — $QKV$ projection, $QK^\top$, scaled softmax with quantization, and $AV$ — onto the same local, registered dataflow, so inter-PE communication is short-range and broadcast-free. The triangular delay, an arithmetic-sequence delay chain, realigns staggered aggregation results back to the systolic rhythm; Welford's online algorithm supplies mean and variance for division-free, square-root-free normalization; and a first-order $2^x$ approximation implements the softmax exponential in shifts and adds.

What would settle it

Take the synthesized bitstream, place it on an Alveo U250 with a real host over PCIe, and measure: if the 400 MHz clock is not met, if board power under the MSA workload exceeds 61.83 W by more than measurement error, or if sustained throughput falls below the predicted 673.4 million tokens/s due to pipeline bubbles or non-overlapped host transfers, the paper's central performance claim is refuted. An independent accuracy check would compare the integerized DeiT-S outputs on chip against the 3-bit quantized model's 77.81% top-1 ImageNet result.

Watch

Extended reading notes

Core claim

The discovery is that the compute-heavy core of DeiT-S multi-head self-attention can be offloaded to an FPGA as a fully systolic, 3-bit integer dataflow without sacrificing accuracy: the authors factor the channel-wise quantization step into a global step, reorder the compute graph so linear layers and matrix multiplications consume low-bit operands directly, and keep normalization and softmax in full precision with only $O(N^2)$ cost. The resulting weight-stationary systolic array, built from 99,072 3-bit MAC processing elements plus array-compatible softmax and layer-normalization units, sustains a 400 MHz clock and delivers 13,568 GOPs/s at 61.83 W on an Alveo U250. Because the design is communication-bound, it needs only a 64-bit/cycle bus (3.13 GB/s) and reaches an operational intensity of 4342 OP/byte, which the authors use to explain why it outperforms prior FPGA implementations and matches or beats same-technology GPUs on power efficiency.

Load-bearing premise

The central performance and efficiency numbers come from Vivado synthesis reports and analytical cycle counts, not from a running FPGA; if the design cannot close timing at 400 MHz, draws more than 61.83 W, or fails to overlap communication with computation in practice, the headline gains over GPUs do not materialize.

Editorial extensions

If this is right

The 3-bit MSA accelerator can process a DeiT-S token every 1.49 µs (673.4 million tokens/s) at 400 MHz with only 3.13 GB/s of bandwidth, so the design fits a PCIe 3.0 x4 link and can be replicated across multiple FPGAs.
The authors' full-model latency estimate is 660.6 µs for DeiT-S (35.63 µs for MSA alone), assuming the MLP runs on a second accelerator and four transmissions per layer; this bounds end-to-end service latency if the estimate holds.
The DSP-free variant trades power efficiency for clock speed: at 500 MHz it reaches 17.37 TOPs/s but only 128.4 GOPs/W, so the DSP-based 400 MHz version remains the better power-efficiency choice.
Under the paper's Dennard-scaling normalization, the design's normalized power efficiency is the highest among the FPGA transformer accelerators evaluated and sits just below NVIDIA A100 and H100 GPUs in normalized throughput.
The 4342 OP/byte operational intensity and 3.13 GB/s bandwidth show that task selection alone can move a transformer workload off the memory-bound part of the roofline model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: since the authors report the design is communication-bound, a straightforward testable extension is to widen the on-chip bus or use HBM; if Equation 16 is right, delivered GOPs/s should rise toward the 42,515 GOPs/s peak without changing the MAC array.
Editorial inference: the same integerized-offload recipe applies to the FFN linear layers, and the authors' own full-model estimate assumes a second accelerator; integrating both on one FPGA would turn the 660.6 µs latency estimate into a measured end-to-end quantity.
Editorial inference: the GPU comparison pits INT3 FPGA MACs against FP32/FP16 GPU cores; benchmarking the same low-bit model with GPU integer or tensor-core instructions is the natural next test and would show whether the efficiency gap persists under matched precision.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The architecture here is genuinely interesting: a 3-bit systolic MSA accelerator with array-compatible softmax and layer normalization, a weight-loading unit, and per-SA pipelining. The module mapping and area/power breakdown are careful, and the OP/byte figure is impressive if it holds up. But the paper's central performance claim does not hold up to its own equations.

The concrete problem: in V-C, Eq. 16 gives the "actual interval" as max(N+d, d/H+2N, (3*N*d)/(64*H)) = 594 cycles for DeiT-S. They then say this gives 673.4 million tokens/s at 400 MHz. That is a 1000x arithmetic error: 400e6/594 = 673,400 tokens/s, not 673.4M. Worse, IV-D says the pipelined accelerator time-multiplexes the six SA heads and the next input waits until the current has gone through all heads, making the steady-state inter-token interval H times the per-head pitch. Eq. 16 uses the per-head communication time (3564/6) as the interval, so even if the arithmetic were right, it would be the wrong quantity. The reported 13,568 GOPs/s and the 1.50x GPU ratio are not derivable from the stated equations without an unstated factor. That is load-bearing.

Other soft spots are minor by comparison: the performance numbers come from synthesis, not a running board; the ImageNet accuracy is carried over from a Q-ViT checkpoint with 20 epochs of finetuning; and the normalization scalars are tuned to match the reference accuracy, so "no observable loss" is partly fitted. The GPU comparisons also mix INT3 FPGA ops with FP32/FP16 GPU ops, which the authors half-acknowledge.

What is actually good: the systolic array-compatible aggregation modules are a real design contribution, and the pipelining/bandwidth tradeoff is clearly explained. The paper is worth engaging, but the headline results need to be re-derived and then measured before they can be believed.

Recommendation: send it to review, but the reviewers should be asked to check the timing math carefully. I would not cite the performance numbers until they are corrected and, ideally, backed by on-board measurement.

Referee Report

5 major / 6 minor

Summary. The paper proposes a model-specialized, systolic-array-based accelerator for 3-bit integerized multi-head self-attention (MSA) in DeiT vision transformers. The authors introduce an integerization that factors step sizes out of matrix multiplications, a systolic array template with array-compatible softmax, layer-normalization, and quantization units, and a time-multiplexed accelerator that executes the six attention heads in one FPGA. They report 96.83% CIFAR-10 and 77.81% ImageNet top-1 accuracy for the integerized DeiT-S, and claim 13,568 GOPs/s, 219.4 GOPs/s/W, and 1.50x throughput / 4.47x power efficiency versus a GTX 1080 on an Alveo U250. All hardware results are reported from Vivado synthesis rather than from a running FPGA.

Significance. If the hardware claims hold, this would be a valuable data point for low-bit transformer inference: a model-specialized INT3 MSA accelerator on a 16 nm FPGA could surpass same-technology GPUs in throughput and power efficiency while retaining quantized DeiT-S accuracy. The systolic-compatible normalization and softmax units, the explicit high operational intensity (4,342 OP/byte), and the PE-level area/power analysis are genuine strengths. However, the central throughput derivation contains a factor-10^3 unit error, the interval formula in Eq. (16) is inconsistent with the pipelining description in Section IV-D, and the headline 13,568 GOPs/s is not derived from the stated PE count and clock. These issues currently prevent the main performance claims from being accepted.

major comments (5)

[Section V-C, Eq. (16)] The conversion from cycles to tokens per second is off by a factor of 10^3. With a 400 MHz clock and a 594-cycle interval, the rate is 400×10^6/594 ≈ 6.73×10^5 tokens/s, i.e., 0.673 million, not '673.4 million tokens per second.' The ideal-case figure has the same problem: 400×10^6/582 ≈ 6.87×10^5, not '684.9 million.' Because these token rates are presented as the throughput of the SA pipeline, this arithmetic error is load-bearing for the paper's throughput claims.
[Section V-C, Eq. (16) versus Section IV-D] The 'actual interval, or 1/throughput' in Eq. (16) uses the per-head amortized communication time (3Nd)/(64H) = 594 cycles as the MSA interval. In the time-multiplexed pipelined design described in Section IV-D, however, the next MSA input must wait until the current input has traversed all six SA stages, so the steady-state period between MSA inputs is governed by the full MSA compute or communication time (thousands of cycles), not by 594 cycles. The inconsistency is already visible in Eq. (15), where the text defines the pitch as max(N+d, d/H+2N) = 582 cycles but then inserts Throughput = 460 cycles to obtain 3,627 cycles. The latency and throughput numbers in Section V-C therefore do not follow uniquely from the stated equations; they need to be re-derived from a single, clearly defined pipeline schedule.
[Section V-H and Table III] The headline 13,568 GOPs/s is not derived anywhere. From the stated 99,072 PEs at 400 MHz, the peak is 99,072×400×10^6 ≈ 3.96×10^13 MAC/s (or ≈7.93×10^13 FLOP/s if a MAC counts as two operations), yet Table III lists a peak of 42,515 GOPs/s and an actual 13,568 GOPs/s with no utilization analysis. Moreover, the table appears to mix MSA-only and full-model figures: the full-model latency estimate in Table IV (660.6 µs for DeiT-S) combined with the listed 4.57 GOPs per inference gives a throughput close to 13,568 GOPs/s, suggesting that the number compared against GPUs may be a full-DeiT-S throughput rather than the MSA accelerator's own throughput. Please state exactly which operations are counted and provide the complete derivation of both 42,515 and 13,568 GOPs/s.
[Section V-B and Abstract] All hardware results are from Vivado synthesis reports, not from a measured FPGA implementation. The abstract says 'We validate the hardware design on a 16nm FPGA,' but the paper reports no place-and-route timing closure, no board-level clock confirmation, no measured power, and no measured throughput. Because Eq. (16) assumes communication/computation overlap with no pipeline bubbles, and because the central comparison depends on a 400 MHz clock and 61.83 W from synthesis, the performance claims require either post-implementation measurement or a clear and consistent caveat that they are projected synthesis estimates.
[Section V-F and Figure 14] The claim that integerization introduces 'no observable loss' is supported by selecting 2^nu = 64 and s = 32 through a sweep over those free parameters, and the ImageNet model is initialized from a Q-ViT checkpoint and finetuned for only 20 epochs. The paper does not report the Q-ViT baseline accuracy against which the loss is measured, nor the variability of the finetuning and parameter-selection procedure. Please report the baseline and the actual accuracy gap (with appropriate confidence intervals) before claiming no observable accuracy loss.

minor comments (6)

[Section V-A] The abstract reports 96.83% CIFAR-10 accuracy, but the experiments section only describes ImageNet training and evaluation; the CIFAR-10 setup and training details should be described, or the claim should be removed.
[Section IV-C] The word 'elabroate' should be 'elaborate.'
[Equation (11)] The first-order exponential approximation is rendered ambiguously; it should read 2^x ≈ x/2 + 1/2.
[Table VI] The column header 'TOPs/s' and the unit 'GOPs/W' are inconsistent with 'GOPs/s/W' used elsewhere; please unify the units across the paper.
[Table V] The column header 'Latency (µs) GOPs' is ambiguous; the values 1.23, 4.57, and 17.54 appear to be per-inference GFLOPs rather than throughput, and the header should state this explicitly.
[Section V-C] Even after correcting the unit error, the text should clarify whether 'tokens' means individual patch tokens or whole images (sequences of N tokens), because the transmission time of 3,564 cycles is for the entire MSA input.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the throughput and accuracy claims are computed from stated cycle intervals, synthesis reports, and external accuracy baselines, rather than from their own conclusions.

full rationale

I find no circular step that reduces a claimed derivation to its own inputs. The throughput and power-efficiency numbers in Section V and Table III are computed from the stated cycle intervals (Eq. 16), the 400 MHz clock, the 61.83 W synthesis-reported power, and the implemented PE count; these are independent quantities, not re-statements of the claimed result. The accuracy claims are empirical comparisons against the external Q-ViT checkpoint [8], with the integerization algebra re-derived in Section III (Eqs. 1-9), so the model-accuracy portion is also self-contained. The choice of 2^nu=64 and s=32 in Section V-F is presented as a parameter sweep to match the previously reported quantized-model accuracy; this is hyperparameter tuning, not a fitted parameter being renamed as a prediction, so it does not satisfy the definition of circularity. Reference [6] is a self-citation for the integerization approach, but the paper describes the method and reports its own fine-tuning experiments, making the citation non-load-bearing. The manuscript does contain caveats about synthesis-based validation and an omitted K-map derivation for the NormQ logic, but those affect confidence in the hardware numbers, not circularity. The apparent unit and interval inconsistencies in Eq. 16 (594 cycles versus 673.4 million tokens/s) are correctness or arithmetic concerns, not a circular reduction, and the hard rules require a specific equation-level reduction to claim circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The hardware result depends mainly on standard digital design assumptions plus three empirical approximations: the global step-size factorization, the first-order exponential approximation, and the two fixed-point normalization scalars chosen by sweep. The model accuracy is largely inherited from Q-ViT [8] and the authors' prior integerization preprint [6]. No new physical entities are introduced.

free parameters (2)

Normalization numerator scaling 2^nu = 64
Chosen by sweeping values in Fig. 14 to keep integerized accuracy close to the 3-bit quantized baseline; fixed-point rounding of 2^nu/i needs enough precision.
Normalization prescale factor s = 32
Also chosen by sweep in Sec. V-F and Fig. 14 to fit each multiplication into the DSP48E2 27x18 multiplier; too small a value degrades accuracy.

assumptions (6)

domain assumption The 3-bit quantized DeiT-S checkpoint from Q-ViT [8] provides the starting point and reference accuracy for ImageNet (77.81%).
Section V-A initializes the integerized model from the pretrained quantized model and finetunes for 20 epochs; the claimed accuracy is not independently established in this paper.
ad hoc to paper Replacing channel-wise step sizes with a global step size introduces no observable accuracy loss.
Required for the factorization in Eq. 9; supported only by the paper's own experiments in Sec. V-F without detailed quantitative comparison.
ad hoc to paper The first-order exponential approximation 2^x approximately x/2 + 1/2 is accurate enough for softmax.
Section IV-C4 states more complex approximations did not significantly improve accuracy, but no experimental data are shown.
domain assumption Synthesis results are representative of actual hardware behavior.
Section V-B explicitly treats Vivado synthesis reports as sufficient validation; no on-board measurement is described.
domain assumption Dennard scaling and TechPowerUp GPU specs allow fair cross-technology comparison.
Section V-I normalizes throughput and power efficiency to other technology nodes; if the scaling model is inaccurate, Figure 16 conclusions weaken.
standard math Welford's online mean and variance computation is numerically suitable in the systolic design.
Used in Eq. 13 and Sec. IV-C5; the algorithm is standard, though fixed-point rounding is analyzed only through the parameter sweep in Sec. V-F.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers." pith.science (2026). https://pith.science/paper/5KS527MC

@misc{pith2026250820334,
  author       = {Pith},
  title        = {Pith review of: Systolic Array-based Architecture for Low-Bit Integerized Vision Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5KS527MC}},
  note         = {Machine review of arXiv:2508.20334}
}

read the original abstract

Transformer-based models are becoming more and more intelligent and are revolutionizing a wide range of human tasks. To support their deployment, AI labs offer inference services that consume hundreds of GWh of energy annually and charge users based on the number of tokens processed. Under this cost model, minimizing power consumption and maximizing throughput have become key design goals for the inference hardware. While graphics processing units (GPUs) are commonly used, their flexibility comes at the cost of low operational intensity and limited efficiency, especially under the high query-per-model ratios of modern inference services. In this work, we address these challenges by proposing a low-bit, model-specialized accelerator that strategically selects tasks with high operation (OP) reuse and minimal communication overhead for offloading. Our design incorporates multiple systolic arrays with deep, fine-grained pipelines and array-compatible units that support essential operations in multi-head self-attention (MSA) module. At the accelerator-level, each self-attention (SA) head is pipelined within a single accelerator to increase data reuse and further minimize bandwidth. Our 3-bit integerized model achieves 96.83% accuracy on CIFAR-10 and 77.81% top-1 accuracy on ImageNet. We validate the hardware design on a 16nm FPGA (Alveo U250), where it delivers 13,568 GigaOps/second (GOPs/s) and 219.4 GOPs/s/W. Compared to a same-technology GPU (GTX 1080), our design offers 1.50x higher throughput and 4.47x better power efficiency. Even against a state-of-the-art GPU (RTX 5090), we still achieve 20% better power efficiency despite having 87% lower throughput.

Figures

Figures reproduced from arXiv: 2508.20334 by the authors.

**Figure 2.** DeiT structure consists of linear projection of flattened patches, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** (A) The original compute graph of ViT shows that floating-point (FP) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (11 more)

**Figure 4.** Figure 4: (A) To reduce communication overhead between the host and FPGA, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 7.** Figure 7: (A) General form of the operation (B) Corresponding timing diagram [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

**Figure 6.** Figure 6: Systolic Array Overview: The template comprises a 2D MAC array, a [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 8.** Figure 8: Comparison between output-stationary and weight-stationary dataflow. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

**Figure 10.** Figure 10: (A) Division-free and Square-root free normalization and quantizer [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]

**Figure 12.** Figure 12: Timing diagram for a SA module. Its throughput depends on two [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

**Figure 13.** Figure 13: Power breakdown: Each section is color-coded or labeled based on [PITH_FULL_IMAGE:figures/full_fig_p009_13.png]

**Figure 14.** Figure 14: The accuracy vs parameters. Lower prescale and numerator results in [PITH_FULL_IMAGE:figures/full_fig_p010_14.png]

**Figure 15.** Figure 15: Accuracy Comparison: We compare our 3-bit integerized model with HG-PIPE [33], Auto-ViT-acc [23], Xiao [34], and I-ViT. Our model achieves lower normalized power at the similar accuracy level, pushing the Pareto frontier out in accuracy-power plot. H. Compare to Same-…

**Figure 16.** Figure 16: Comparison of normalized power efficiency and throughput. Our [PITH_FULL_IMAGE:figures/full_fig_p012_16.png]

**Figure 17.** Figure 17: (A) High-throughput normalization and quantization: The pipelined [PITH_FULL_IMAGE:figures/full_fig_p013_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 24 canonical work pages

[1]

Chatgpt consumes enough power in one year to charge over three million electric cars,

I. Todorovic, “Chatgpt consumes enough power in one year to charge over three million electric cars,” Balkan Green Energy News . [On- line]. Available: https://balkangreenenergynews.com/chatgpt-consumes- enough-power-in-one-year-to-charge-over-three-million-electric-cars/

work page
[2]

Openai training and inference costs could reach 7bnfor2024,aistartupsettolose 5bn - report,

S. Moss, “Openai training and inference costs could reach 7bnfor2024,aistartupsettolose 5bn - report,” Data Center Dynamices . [Online]. Available: https: //www.datacenterdynamics.com/en/news/openai-training-and-inference- costs-could-reach-7bn-for-2024-ai-startup-set-to-lose-5bn-report/

work page 2024
[3]

From words to watts: Benchmarking the energy costs of large language model infer- ence,

S. Samsi, D. Zhao, J. McDonald, B. Li, A. Michaleas, M. Jones, W. Bergeron, J. Kepner, D. Tiwari, and V . Gadepally, “From words to watts: Benchmarking the energy costs of large language model infer- ence,” in 2023 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2023, pp. 1–9

2023
[4]

Beyond chinchilla- optimal: Accounting for inference in language model scaling laws,

N. Sardana, J. Portes, S. Doubov, and J. Frankle, “Beyond chinchilla- optimal: Accounting for inference in language model scaling laws,” arXiv preprint arXiv:2401.00448 , 2023

arXiv 2023
[5]

Quantization and training of neural networks for efficient integer-arithmetic-only inference,

B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704– 2713

2018
[6]

Low-Bit Integerization of Vision Transformers using Operand Reordering for Efficient Hardware

C.-Y . Lin and S. Shah, “Low-bit integerization of vision transform- ers using operand reodering for efficient hardware,” arXiv preprint arXiv:2504.18547, 2025

work page Pith review arXiv 2025
[7]

I-vit: Integer-only quantization for efficient vision transformer inference,

Z. Li and Q. Gu, “I-vit: Integer-only quantization for efficient vision transformer inference,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 065–17 075

2023
[8]

Q-vit: Fully differentiable quantization for vision transformer,

Z. Li, T. Yang, P. Wang, and J. Cheng, “Q-vit: Fully differentiable quantization for vision transformer,” arXiv preprint arXiv:2201.07703 , 2022

arXiv 2022

Show all 44 references

[9]

Roofline: an insightful visual performance model for multicore architectures,

S. Williams, A. Waterman, and D. Patterson, “Roofline: an insightful visual performance model for multicore architectures,” Communications of the ACM , vol. 52, no. 4, pp. 65–76, 2009

2009
[10]

Thinet: A filter level pruning method for deep neural network compression,

J.-H. Luo, J. Wu, and W. Lin, “Thinet: A filter level pruning method for deep neural network compression,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 5058–5066

2017
[11]

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,

S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149 , 2015

2015 arXiv
[12]

Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1,

M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y . Ben- gio, “Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1,” arXiv preprint arXiv:1602.02830, 2016

2016 arXiv
[13]

Xnor-net: Imagenet classification using binary convolutional neural networks,

M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi, “Xnor-net: Imagenet classification using binary convolutional neural networks,” in European conference on computer vision. Springer, 2016, pp. 525–542

2016
[14]

Ternary neural networks for resource-efficient ai applications,

H. Alemdar, V . Leroy, A. Prost-Boucle, and F. P ´etrot, “Ternary neural networks for resource-efficient ai applications,” in 2017 international joint conference on neural networks (IJCNN) . IEEE, 2017, pp. 2547– 2554

2017
[15]

Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,

S. Zhou, Y . Wu, Z. Ni, X. Zhou, H. Wen, and Y . Zou, “Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016

2016 arXiv
[16]

Bridging the accuracy gap for 2-bit quantized neural networks (qnn),

J. Choi, P. I.-J. Chuang, Z. Wang, S. Venkataramani, V . Srinivasan, and K. Gopalakrishnan, “Bridging the accuracy gap for 2-bit quantized neural networks (qnn),” arXiv preprint arXiv:1807.06964 , 2018

2018 arXiv
[17]

Fq-vit: Post-training quantization for fully quantized vision transformer,

Y . Lin, T. Zhang, P. Sun, Z. Li, and S. Zhou, “Fq-vit: Post-training quantization for fully quantized vision transformer,” arXiv preprint arXiv:2111.13824, 2021

2021 arXiv
[18]

I-bert: Integer-only bert quantization,

S. Kim, A. Gholami, Z. Yao, M. W. Mahoney, and K. Keutzer, “I-bert: Integer-only bert quantization,” in International conference on machine learning. PMLR, 2021, pp. 5506–5518

2021
[19]

An integer- only and group-vector systolic accelerator for efficiently mapping vision transformer on edge,

M. Huang, J. Luo, C. Ding, Z. Wei, S. Huang, and H. Yu, “An integer- only and group-vector systolic accelerator for efficiently mapping vision transformer on edge,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 70, no. 12, pp. 5289–5301, 2023

2023
[20]

Accelerating attention mechanism on fpgas based on efficient reconfigurable systolic array,

W. Ye, X. Zhou, J. Zhou, C. Chen, and K. Li, “Accelerating attention mechanism on fpgas based on efficient reconfigurable systolic array,” ACM Transactions on Embedded Computing Systems , vol. 22, no. 6, pp. 1–22, 2023

2023
[21]

Calabash: Accelerating attention using a systolic array chain on fpgas,

Z. Luo, L. Lu, Y . Jin, L. Jia, and Y . Liang, “Calabash: Accelerating attention using a systolic array chain on fpgas,” in 2023 33rd Inter- national Conference on Field-Programmable Logic and Applications (FPL). IEEE, 2023, pp. 242–247

2023
[22]

A 109-gops/w fpga-based vision transformer accelerator with weight-loop dataflow featuring data reusing and resource saving,

Y . Zhang, L. Feng, H. Shan, and Z. Zhu, “A 109-gops/w fpga-based vision transformer accelerator with weight-loop dataflow featuring data reusing and resource saving,” IEEE Transactions on Circuits and Sys- tems for Video Technology, 2024

2024
[23]

Auto-vit-acc: An fpga- aware automatic acceleration framework for vision transformer with mixed-scheme quantization,

Z. Li, M. Sun, A. Lu, H. Ma, G. Yuan, Y . Xie, H. Tang, Y . Li, M. Leeser, Z. Wang, X. Lin, and F. Zhenman, “Auto-vit-acc: An fpga- aware automatic acceleration framework for vision transformer with mixed-scheme quantization,” in 2022 32nd International Conference on Field-Pro...

2022
[24]

Me-vit: A single-load memory-efficient fpga accelerator for vision transformers,

K. Marino, P. Zhang, and V . K. Prasanna, “Me-vit: A single-load memory-efficient fpga accelerator for vision transformers,” in2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC) . IEEE, 2023, pp. 213–223

2023
[25]

Eyeriss: An energy- efficient reconfigurable accelerator for deep convolutional neural net- works,

Y .-H. Chen, T. Krishna, J. S. Emer, and V . Sze, “Eyeriss: An energy- efficient reconfigurable accelerator for deep convolutional neural net- works,” IEEE journal of solid-state circuits, vol. 52, no. 1, pp. 127–138, 2016

2016
[26]

An image is worth 16x16 words: Transform- ers for image recognition at scale,

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and H. Neil, “An image is worth 16x16 words: Transform- ers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

2010 arXiv
[27]

Training data-efficient image transformers & distillation through attention,

H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distillation through attention,” in International conference on machine learning . PMLR, 2021, pp. 10 347–10 357

2021
[28]

Fine-grained hardware switching scheme for power reduction in multiplication,

Y . Huang, C. Li, M. Li, L. Van der Perre, and W. Dehaene, “Fine-grained hardware switching scheme for power reduction in multiplication,” Electronics Letters , vol. 52, no. 16, pp. 1374–1375,
[29]

Note on a method for calculating corrected sums of squares and products,

B. P. Welford, “Note on a method for calculating corrected sums of squares and products,” Technometrics, vol. 4, no. 3, pp. 419–420, 1962

1962
[30]

Ai and memory wall,

A. Gholami, Z. Yao, S. Kim, C. Hooper, M. W. Mahoney, and K. Keutzer, “Ai and memory wall,” IEEE Micro , vol. 44, no. 3, pp. 33–39, 2024

2024
[31]

Low power wallace tree multiplier using modified full adder,

K. B. Jaiswal, N. Kumar, P. Seshadri, and L. G, “Low power wallace tree multiplier using modified full adder,” in 2015 3rd international con- ference on signal processing, communication and networking (ICSCN) . IEEE, 2015, pp. 1–4

2015
[32]

Design and imple- mentation of wallace compressor multiplier using vedic mathematics,

P. R. Sarangpure, D. S. Chaudhari, and Y . D. Kapse, “Design and imple- mentation of wallace compressor multiplier using vedic mathematics,” 2017

2017
[33]

Hg-pipe: Vision transformer acceleration with hybrid-grained pipeline,

Q. Guo, J. Wan, S. Xu, M. Li, and Y . Wang, “Hg-pipe: Vision transformer acceleration with hybrid-grained pipeline,” in Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, 2024, pp. 1–9

2024
[34]

Refining datapath for microscaling vits,

C. Xiao, J. Cheng, and A. Zhao, “Refining datapath for microscaling vits,” arXiv preprint arXiv:2505.22194 , 2025

2025 arXiv
[35]

Via: A novel vision-transformer accelerator based on fpga,

T. Wang, L. Gong, C. Wang, Y . Yang, Y . Gao, X. Zhou, and H. Chen, “Via: A novel vision-transformer accelerator based on fpga,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 11, pp. 4088–4099, 2022

2022
[36]

Dfx: A low-latency multi-fpga appliance for accelerating transformer-based text generation,

S. Hong, S. Moon, J. Kim, S. Lee, M. Kim, D. Lee, and J.-Y . Kim, “Dfx: A low-latency multi-fpga appliance for accelerating transformer-based text generation,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 2022, pp. 616–630

2022
[37]

Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design,

H. Fan, T. Chau, S. I. Venieris, R. Lee, A. Kouris, W. Luk, N. D. Lane, and M. S. Abdelfattah, “Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 202...

2022
[38]

The annotated transformer,

A. M. Rush, “The annotated transformer,” in Proceedings of workshop for NLP open source software (NLP-OSS) , 2018, pp. 52–60

2018
[39]

Heatvit: Hardware-efficient adaptive token pruning for vision transformers,

P. Dong, M. Sun, A. Lu, Y . Xie, K. Liu, Z. Kong, X. Meng, Z. Li, X. Lin, Z. Fang, and Y . Wang, “Heatvit: Hardware-efficient adaptive token pruning for vision transformers,” in 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2023, pp. 442–455

2023
[40]

Hardware accelerator for multi-head attention and position-wise feed-forward in the trans- former,

S. Lu, M. Wang, S. Liang, J. Lin, and Z. Wang, “Hardware accelerator for multi-head attention and position-wise feed-forward in the trans- former,” in 2020 IEEE 33rd International System-on-Chip Conference (SOCC). IEEE, 2020, pp. 84–89

2020
[41]

Ftrans: energy-efficient acceleration of transformers using fpga,

B. Li, S. Pandey, H. Fang, Y . Lyv, J. Li, J. Chen, M. Xie, L. Wan, H. Liu, and C. Ding, “Ftrans: energy-efficient acceleration of transformers using fpga,” in Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design , 2020, pp. 175–180

2020
[42]

Design of ion-implanted mosfet’s with very small physical dimensions,

R. H. Dennard, F. H. Gaensslen, H.-N. Yu, V . L. Rideout, E. Bassous, and A. R. LeBlanc, “Design of ion-implanted mosfet’s with very small physical dimensions,” IEEE Journal of solid-state circuits , vol. 9, no. 5, pp. 256–268, 2003

2003
[43]

Techpowerup,

“Techpowerup,” https://www.techpowerup.com/
[2016]

Available: https://ietresearch.onlinelibrary.wiley.com/ doi/abs/10.1049/el.2015.3828

[Online]. Available: https://ietresearch.onlinelibrary.wiley.com/ doi/abs/10.1049/el.2015.3828

2015

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Chatgpt consumes enough power in one year to charge over three million electric cars,

I. Todorovic, “Chatgpt consumes enough power in one year to charge over three million electric cars,” Balkan Green Energy News . [On- line]. Available: https://balkangreenenergynews.com/chatgpt-consumes- enough-power-in-one-year-to-charge-over-three-million-electric-cars/

work page

[2] [2]

Openai training and inference costs could reach 7bnfor2024,aistartupsettolose 5bn - report,

S. Moss, “Openai training and inference costs could reach 7bnfor2024,aistartupsettolose 5bn - report,” Data Center Dynamices . [Online]. Available: https: //www.datacenterdynamics.com/en/news/openai-training-and-inference- costs-could-reach-7bn-for-2024-ai-startup-set-to-lose-5bn-report/

work page 2024

[3] [3]

From words to watts: Benchmarking the energy costs of large language model infer- ence,

S. Samsi, D. Zhao, J. McDonald, B. Li, A. Michaleas, M. Jones, W. Bergeron, J. Kepner, D. Tiwari, and V . Gadepally, “From words to watts: Benchmarking the energy costs of large language model infer- ence,” in 2023 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2023, pp. 1–9

2023

[4] [4]

Beyond chinchilla- optimal: Accounting for inference in language model scaling laws,

N. Sardana, J. Portes, S. Doubov, and J. Frankle, “Beyond chinchilla- optimal: Accounting for inference in language model scaling laws,” arXiv preprint arXiv:2401.00448 , 2023

arXiv 2023

[5] [5]

Quantization and training of neural networks for efficient integer-arithmetic-only inference,

B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704– 2713

2018

[6] [6]

Low-Bit Integerization of Vision Transformers using Operand Reordering for Efficient Hardware

C.-Y . Lin and S. Shah, “Low-bit integerization of vision transform- ers using operand reodering for efficient hardware,” arXiv preprint arXiv:2504.18547, 2025

work page Pith review arXiv 2025

[7] [7]

I-vit: Integer-only quantization for efficient vision transformer inference,

Z. Li and Q. Gu, “I-vit: Integer-only quantization for efficient vision transformer inference,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 065–17 075

2023

[8] [8]

Q-vit: Fully differentiable quantization for vision transformer,

Z. Li, T. Yang, P. Wang, and J. Cheng, “Q-vit: Fully differentiable quantization for vision transformer,” arXiv preprint arXiv:2201.07703 , 2022

arXiv 2022