REVIEW 3 major objections 5 minor 1 cited by
OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read OpenGeMM is a programmable GeMM generator that reports 81.89%–99.34% array utilization on real DNN workloads and 3.58–16.40x normalized throughput gains over the baseline system.
desk verdict Useful open-source GeMM accelerator with credible utilization results; the speedup claim needs a fairer baseline and full-system accounting. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the 3D MAC array, an $(M_u, N_u)$ mesh of $K_u$-length dot-product units that spatially unrolls all three dimensions of a GeMM tile. Each dot-product unit accumulates a partial sum in an output-stationary register, while A and B vectors are broadcast across rows and columns to maximize data reuse. Around this core, the load-bearing mechanisms are the pre-loadable CSR configuration registers, the producer-consumer pre-fetch buffers, and the strided address generator units (AGUs) that place streams to avoid multi-banked scratchpad contention.
What would settle it
Rerun the OpenGeMM RTL simulations while including all off-chip DRAM-to-SRAM transfer cycles in the execution-time count, and run the baseline generator on the same technology node with the same configuration; if utilization falls below the reported 81.89%–99.34% range or the normalized throughput advantage drops toward 1x, the central claim fails.
Extended reading notes
Core claim
The central claim is that a GeMM accelerator can be made to run real DNN workloads at near-peak array utilization through careful coupling of a small RISC-V core and a multi-banked scratchpad. Specifically, the paper demonstrates that three mechanisms—configuration pre-loading (overlapping CSR programming with compute), input pre-fetch with output buffering (keeping data in flight so the MAC array never waits), and programmable strided address generation (avoiding scratchpad bank conflicts)—together raise median utilization 2.78x over a baseline version of the same platform. On four full models, the measured overall utilization spans 81.89% (MobileNetV2) to 99.34% (BERT-Base), and the platform achieves 4.68 TOPS/W.
Load-bearing premise
The reported utilization and speedup numbers count only on-chip cycles and use published performance data from a different technology node for the baseline, so if off-chip memory traffic were included in the cycle count or the baseline were run under identical conditions, the headline gains could change.
Editorial extensions
If this is right
- The same generated 8x8x8 array reaches 96% or higher utilization on ResNet18 and Transformer workloads, meaning one fixed tiling can cover both CNN and Transformer inference without per-model redesign.
- The ablation study attributes a 2.78x median utilization improvement to the three mechanisms, so the mechanisms transfer to other accelerator generators as a recipe for hiding control and memory latency.
- Because programming happens through standard RISC-V CSR instructions with a pre-loading scheme, software can switch layer configurations without stalling compute, keeping the core busy across layer boundaries.
- With a control core occupying roughly 1.13% of system area and 2.4% of power, the result implies that programmability costs little when the memory system is coupled tightly to the array.
- The reported 4.68 TOPS/W system efficiency and best operation-area efficiency among compared int8 platforms suggest the design is competitive for edge deployment at 200 MHz.
Reading between the lines
- A natural testable extension is to count off-chip DRAM-to-SRAM cycles in the utilization metric; if the reported 81.89%–99.34% holds with those cycles included, OpenGeMM's claim would be even stronger.
- The strided-address mechanism implies a compiler-level layout optimizer could be built to choose bank-aware strides automatically for arbitrary matrix shapes, extending the work beyond hand-tiled workloads.
- The same pre-loading and buffering recipe could be applied to other streaming accelerators, such as convolvers or attention engines, where configuration latency and memory stalls are the main utilization limiters.
- The reported power and area figures come from a single block-matrix workload at one voltage and frequency point; full-model energy measurements would clarify whether the 4.68 TOPS/W figure holds in end-to-end inference.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. OpenGeMM proposes an open-source, parameterized Chisel-based GeMM accelerator generator integrated with a lightweight RISC-V Snitch core and a tightly coupled multi-banked scratchpad memory. Three system-level mechanisms—configuration pre-loading, input prefetching with output buffering, and programmable strided memory access—are introduced to maintain high MAC array utilization. The paper reports cycle-accurate RTL simulation results showing 81.89%–99.34% overall utilization across MobileNetV2, ResNet18, ViT-B-16, and BERT-Base, and claims 3.58×–16.40× normalized throughput speedup over Gemmini as well as 4.68 TOPS/W system efficiency, with area and power breakdowns from TSMC 16nm synthesis.
Significance. If the headline claims hold, OpenGeMM is a valuable open-source contribution: the generator approach, the lightweight control integration, and the three utilization-enhancement mechanisms are well motivated, and the RTL-simulation-based ablation study is a credible methodology for the core utilization claim. The open-source release and the parameterized Chisel generator are concrete strengths that will enable reproduction and extension. However, the headline speedup and system-efficiency numbers depend on the comparison boundary chosen in Footnote 2 and on an external Gemmini baseline; these dependencies need to be made precise and quantified before the central performance comparison can be accepted as stated.
major comments (3)
- [Section 4.3, Footnote 2] The footnote states that DRAM-to-SRAM data movement cycles are not counted. Consequently, the utilization and cycle counts in Table 2 are compute-plus-SPM metrics, not end-to-end execution times, while the abstract and Section 4.4 present the 3.58×–16.40× speedup and 4.68 TOPS/W as system-level results without this caveat. This is load-bearing: for edge workloads, off-chip fill time can dominate when the SPM is limited. Please report end-to-end cycles including DMA/DRAM fill, or provide a quantitative sensitivity analysis that shows how the speedup and TOPS/W change under realistic DRAM bandwidth assumptions.
- [Table 3 and Figure 7] The Gemmini baseline is taken from [32], a 22nm FinFET SoC running at 1 GHz with 512 GOPS peak, while OpenGeMM is synthesized in TSMC 16nm at 200 MHz. The area-normalized throughput comparison (GOPS/mm^2) therefore mixes process node, frequency, and memory-accounting conventions, and the unqualified 'speedup' wording overstates the comparison. Please either run Gemmini under the same simulation flow and memory-accounting boundary, or clearly label the result as an estimated cross-implementation comparison and remove the unqualified 'speedup' language. Additionally, specify where the 'average temporal utilization on average 6.25%' for Gemmini comes from; if it is taken from [32], give the workloads and Gemmini configuration used.
- [Section 4.4] The 4.68 TOPS/W efficiency is computed from a total power of 43.8 mW, but the power breakdown in Figure 6 appears to include only on-chip components (SPM, GeMM core, instruction cache, streamers). If DRAM access power is excluded, the metric should be labeled as on-chip system efficiency, or the DRAM power should be included. This is important because Table 3 compares 'system efficiencies' across accelerators, and inconsistent power boundaries make the comparison misleading.
minor comments (5)
- [Section 4.2] The text says 'input data pre-infecting'; this should be 'pre-fetching'.
- [Section 4.3] The phrase 'tick channels' should likely be 'thin channels'; depthwise convolutions have few input channels, which explains the lower spatial utilization.
- [Figure 7 caption] The caption says 'normilized throughput'; it should be 'normalized throughput'.
- [Table 3] OpenGeMM's area is listed as 0.62 mm^2 in the table while Section 4.4 reports a cell area of 0.531 mm^2; the footnote explains this as layout area with 60% cell density, but the text should clarify which area value is used in the GOPS/mm^2 comparison.
- [Table 2] The definition of 'overall utilization' (OU) is not explicitly given; stating the formula (e.g., OU = SU × TU or cycle-based definition) would make the metric reproducible.
Circularity Check
No significant circularity: the headline utilization, speedup, and efficiency numbers are empirical RTL/power measurements, not derived from the claims they support.
full rationale
OpenGeMM's central claims are measured rather than derived. Utilization (81.89% to 99.34%) is a direct cycle count from Verilator RTL simulation of the generated accelerator, and the ablation isolates the three mechanisms against a baseline implementation. The 3.58x to 16.40x normalized throughput speedup versus Gemmini uses external performance data from reference [32], with the fairness caveat that process nodes, memory accounting, and configuration differ; that is a benchmarking concern, not a circular reduction. The 4.68 TOPS/W figure is computed as measured peak performance (204.8 GOPS) divided by measured power (43.8 mW), an empirical efficiency metric, not a prediction identical to an input. The only self-citations are design-space exploration and prior system references (e.g., ZigZag [20], NNASim [18], CMDS [17]), and these are used to motivate design choices that are also supported by independent reasoning in the text, such as the partial-sum precision argument for output-stationary dataflow. None of the measured results is fitted to the claim or defined in terms of the claim, so no step in the derivation chain is circular.
Assumptions & free parameters
free parameters (4)
- GeMM array dimensions (Mu, Nu, Ku) =
8x8x8
- Prefetch and output buffer depth =
2 for the main comparison; 3 and 4 explored
- Integer precisions (PA, PB, PC) =
8, 8, 32
- Clock frequency and supply voltage =
200 MHz, 0.675 V
assumptions (5)
- domain assumption Output-stationary dataflow is preferable to weight-stationary for GeMM because partial sums have higher precision than weights.
- domain assumption The multi-banked scratchpad with 16 read ports and 32 write ports provides sufficient bandwidth so that the measured utilization is limited by compute and streamer behavior, not by memory port contention.
- ad hoc to paper Off-chip DRAM-to-SRAM data movement cycles can be excluded when reporting performance and efficiency.
- domain assumption Gemmini's published performance data from [32] are a fair baseline for normalized throughput comparison.
- domain assumption Synthesis and power analysis at 200MHz, 0.675V in 16nm, with 60% cell density layout estimation, predict real silicon behavior.
Cite this review
Pith. "Pith review of OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling." pith.science (2026). https://pith.science/paper/WH7LRBUT
@misc{pith2026241109543,
author = {Pith},
title = {Pith review of: OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling},
year = {2026},
howpublished = {\url{https://pith.science/paper/WH7LRBUT}},
note = {Machine review of arXiv:2411.09543}
}
read the original abstract
Deep neural networks (DNNs) face significant challenges when deployed on resource-constrained extreme edge devices due to their computational and data-intensive nature. While standalone accelerators tailored for specific application scenarios suffer from inflexible control and limited programmability, generic hardware acceleration platforms coupled with RISC-V CPUs can enable high reusability and flexibility, yet typically at the expense of system level efficiency and low utilization. To fill this gap, we propose OpenGeMM, an open-source acceleration platform, jointly demonstrating high efficiency and utilization, as well as ease of configurability and programmability. OpenGeMM encompasses a parameterized Chisel-coded GeMM accelerator, a lightweight RISC-V processor, and a tightly coupled multi-banked scratchpad memory. The GeMM core utilization and system efficiency are boosted through three mechanisms: configuration pre-loading, input pre-fetching with output buffering, and programmable strided memory access. Experimental results show that OpenGeMM can consistently achieve hardware utilization ranging from 81.89% to 99.34% across diverse CNN and Transformer workloads. Compared to the SotA open-source Gemmini accelerator, OpenGeMM demonstrates a 3.58x to 16.40x speedup on normalized throughput across a wide variety ofGeMM workloads, while achieving 4.68 TOPS/W system efficiency.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Controllable Latent Space Augmentation for Digital Pathology
A digital-pathology augmentation method is announced in the abstract, but the submitted full text is a different paper about computer hardware, making the claims unauditable.
Reference graph
Works this paper leans on
-
[32]
A 16mm 2 106.1 gops/w heterogeneous risc-v multi-core multi-accelerator soc in low-power 22nm finfet
Abraham Gonzalez, Jerry Zhao, Ben Korpan, Hasan Genc, Colin Schmidt, John Wright, Ayan Biswas, Alon Amid, Farhana Sheikh, Anton Sorokin, et al. A 16mm 2 106.1 gops/w heterogeneous risc-v multi-core multi-accelerator soc in low-power 22nm finfet. In ESSCIRC 2021-IEEE 47th European Solid State Circuits Conference (ESSCIRC), pages 259–262. IEEE, 2021
work page 2021
-
[1]
Tinyml for ultra-low power ai and large scale iot deployments: A systematic review
Nikolaos Schizas, Aristeidis Karras, Christos Karras, and Spyros Sioutas. Tinyml for ultra-low power ai and large scale iot deployments: A systematic review. Future Internet, 14(12):363, 2022
work page 2022
-
[2]
A survey on deep learning hardware accelerators for heterogeneous hpc platforms, 2023
Cristina Silvano, Daniele Ielmini, Fabrizio Ferrandi, Leandro Fiorin, Serena Curzel, Luca Benini, Francesco Conti, Angelo Garofalo, Cristian Zambelli, Enrico Calore, Sebastiano Fabio Schifano, Maurizio Palesi, Giuseppe Ascia, Davide Patti, Stefania Perri, Nicola Petra, Davide De Caro, Luciano Lavagno, Teodoro Urso, Valeria Cardellini, Gian Carlo Cardarill...
work page 2023
-
[3]
A survey of accelerator architectures for deep neural networks
Yiran Chen, Yuan Xie, Linghao Song, Fan Chen, and Tianqi Tang. A survey of accelerator architectures for deep neural networks. Engineering, 6(3):264–274, 2020
work page 2020
-
[4]
Fast and scalable multicore yolov3-tiny accelerator using input stationary systolic architecture
Trio Adiono, Rhesa Muhammad Ramadhan, Nana Sutisna, Infall Syafalni, Rah- mat Mulyawan, and Chang-Hong Lin. Fast and scalable multicore yolov3-tiny accelerator using input stationary systolic architecture. IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2023
work page 2023
-
[5]
Depfin: A 12nm, 3.8 tops depth-first cnn processor for high res
Koen Goetschalckx and Marian Verhelst. Depfin: A 12nm, 3.8 tops depth-first cnn processor for high res. image processing. In2021 Symposium on VLSI Circuits, pages 1–2. IEEE, 2021
work page 2021
-
[6]
Aˆ 3: Accelerating attention mechanisms in neural networks with approximation
Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W Lee, et al. Aˆ 3: Accelerating attention mechanisms in neural networks with approximation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 328–341. IEEE, 2020
work page 2020
-
[7]
The nvidia deep learning accelerator
Frans Sijstermans. The nvidia deep learning accelerator. In Hot Chips, volume 30, pages 19–21, 2018
work page 2018
Show all 32 references
-
[8]
The implementa- tion of lenet-5 with nvdla on risc-v soc
Shanggong Feng, Junning Wu, Shengang Zhou, and Renwei Li. The implementa- tion of lenet-5 with nvdla on risc-v soc. In2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS) , pages 39–42. IEEE, 2019
2019
-
[9]
A chipyard comparison of nvdla and gemmini
Abraham Gonzalez and Charles Hong. A chipyard comparison of nvdla and gemmini. Berkeley, CA, USA, Tech. Rep. EE, pages 290–2, 2020
2020
-
[10]
The risc- v compressed instruction set manual, version 1.7
Andrew Waterman, Yunsup Lee, David A Patterson, and Krste Asanović. The risc- v compressed instruction set manual, version 1.7. EECS Department, University of California, Berkeley, UCB/EECS-2015-157, 2015
2015
-
[11]
Marsellus: A heterogeneous risc-v ai-iot end-node soc with 2–8 b dnn acceleration and 30 IEEE Journal of Solid-State Circuits , 59(1):128–142, January 2024
Francesco Conti, Gianna Paulin, Angelo Garofalo, Davide Rossi, Alfio Di Mauro, Georg Rutishauser, Gianmarco Ottavi, Manuel Eggiman, Hayate Okuhara, and Luca Benini. Marsellus: A heterogeneous risc-v ai-iot end-node soc with 2–8 b dnn acceleration and 30 IEEE Journal of Solid-S...
2024
-
[12]
Gem- mini: Enabling systematic deep-learning architecture evaluation via full-stack integration
Hasan Genc, Seah Kim, Alon Amid, Ameer Haj-Ali, Vighnesh Iyer, Pranav Prakash, Jerry Zhao, Daniel Grubb, Harrison Liew, Howard Mao, et al. Gem- mini: Enabling systematic deep-learning architecture evaluation via full-stack integration. In 2021 58th ACM/IEEE Design Automation C...
2021
-
[13]
Conna: Configurable matrix multiplication engine for neural network acceleration
Sang-Soo Park and Ki-Seok Chung. Conna: Configurable matrix multiplication engine for neural network acceleration. Electronics, 11(15), 2022. ISSN 2079-9292. doi: 10.3390/electronics11152373. URL https://www.mdpi.com/2079-9292/11/15/ 2373
2022 doi
-
[14]
Xnor neural engine: A hardware accelerator ip for 21.6-fj/op binary neural network inference
Francesco Conti, Pasquale Davide Schiavone, and Luca Benini. Xnor neural engine: A hardware accelerator ip for 21.6-fj/op binary neural network inference. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37 (11):2940–2951, November 2018. ISSN 1937...
2018
-
[15]
Redmule: A mixed-precision matrix-matrix operation engine for flexible and energy-efficient on-chip linear algebra and tinyml training acceleration
Yvan Tortorella, Luca Bertaccini, Luca Benini, Davide Rossi, and Francesco Conti. Redmule: A mixed-precision matrix-matrix operation engine for flexible and energy-efficient on-chip linear algebra and tinyml training acceleration. arXiv preprint arXiv:2301.03904, 2023
2023 arXiv
-
[16]
Designing domain- specific heterogeneous architectures from dataflow programs
Süleyman Savas, Zain Ul-Abdin, and Tomas Nordström. Designing domain- specific heterogeneous architectures from dataflow programs. Computers, 7(2): 27, 2018
2018
-
[17]
Cmds: Cross-layer dataflow op- timization for dnn accelerators exploiting multi-bank memories
Man Shi, Steven Colleman, Charlotte VanDeMieroop, Antony Joseph, Maurice Meijer, Wim Dehaene, and Marian Verhelst. Cmds: Cross-layer dataflow op- timization for dnn accelerators exploiting multi-bank memories. In 2023 24th International Symposium on Quality Electronic Design (...
2023
-
[18]
Nnasim: An efficient event-driven simulator for dnn accelerators with accurate timing and area models
Xiaoling Yi, Jiangnan Yu, Zheng Wu, Xiankui Xiong, Dong Xu, Chixiao Chen, Jun Tao, and Fan Yang. Nnasim: An efficient event-driven simulator for dnn accelerators with accurate timing and area models. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS) , pages ...
2022
-
[19]
Sarda, Vikram Jain, Kodai Ueyoshi, Ioannis A
Pouya Houshmand, Giuseppe M. Sarda, Vikram Jain, Kodai Ueyoshi, Ioannis A. Papistas, Man Shi, Qilin Zheng, Debjyoti Bhattacharjee, Arindam Mallik, Peter Debacker, Diederik Verkest, and Marian Verhelst. Diana: An end-to-end hybrid digital and analog neural network soc for the e...
2023
-
[20]
Zigzag: Enlarging joint architecture-mapping design space exploration for dnn accelerators
Linyan Mei, Pouya Houshmand, Vikram Jain, Sebastian Giraldo, and Marian Verhelst. Zigzag: Enlarging joint architecture-mapping design space exploration for dnn accelerators. IEEE Transactions on Computers , 70(8):1160–1174, 2021
2021
-
[21]
Low- memory gemm-based convolution algorithms for deep neural networks
Andrew Anderson, Aravind Vasudevan, Cormac Keane, and David Gregg. Low- memory gemm-based convolution algorithms for deep neural networks. arXiv preprint arXiv:1709.03395, 2017
2017 arXiv
-
[22]
Snitch: A tiny pseudo dual-issue processor for area and energy efficient execution of floating- point intensive workloads
Florian Zaruba, Fabian Schuiki, Torsten Hoefler, and Luca Benini. Snitch: A tiny pseudo dual-issue processor for area and energy efficient execution of floating- point intensive workloads. IEEE Transactions on Computers , 70(11):1845–1860, 2020
2020
-
[23]
The RISC-V Instruction Set Manual, Volume II: Privileged Architecture
Andrew Waterman, Krste Asanović, and John Hauser. The RISC-V Instruction Set Manual, Volume II: Privileged Architecture. RISC-V International, December 2021. Document Version 20211203
2021
-
[24]
Stream semantic registers: A lightweight risc-v isa extension achieving full compute utilization in single-issue cores
Fabian Schuiki, Florian Zaruba, Torsten Hoefler, and Luca Benini. Stream semantic registers: A lightweight risc-v isa extension achieving full compute utilization in single-issue cores. IEEE Transactions on Computers , 70(2):212–227, 2020
2020
-
[25]
Chisel: constructing hardware in a scala embedded language
Jonathan Bachrach, Huy Vo, Brian Richards, Yunsup Lee, Andrew Waterman, Rimas Avižienis, John Wawrzynek, and Krste Asanović. Chisel: constructing hardware in a scala embedded language. In Proceedings of the 49th Annual Design Automation Conference, pages 1216–1225, 2012
2012
-
[26]
Sigma: A sparse and irregular gemm accelerator with flexible interconnects for dnn training
Eric Qin, Ananda Samajdar, Hyoukjun Kwon, Vineet Nadella, Sudarshan Srini- vasan, Dipankar Das, Bharat Kaul, and Tushar Krishna. Sigma: A sparse and irregular gemm accelerator with flexible interconnects for dnn training. In 2020 IEEE International Symposium on High Performanc...
2020
-
[27]
Soft tiles: Capturing physical implementation flexibility for tightly-coupled parallel processing clusters
Gianna Paulin, Matheus Cavalcante, Paul Scheffler, Luca Bertaccini, Yichao Zhang, Frank Gürkaynak, and Luca Benini. Soft tiles: Capturing physical implementation flexibility for tightly-coupled parallel processing clusters. In 2022 IEEE Computer Society Annual Symposium on VLS...
2022
-
[28]
Identity mappings in deep residual networks
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 , pages 630–645. Springer, 2016
2016
-
[29]
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition , pages 4510–4520, 2018
2018
-
[30]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arX...
2010 arXiv
-
[31]
Bert: Pre- training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018
2018 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.