Pith. sign in

REVIEW 3 major objections 5 minor 79 references

Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HyFlexPIM shows after SVD fine-tuning, 5-10% of weights in SLC and the rest in 2-bit MLC keep accuracy loss under 1%, decoder loss under 10%, and deliver up to 1.86x throughput and 1.45x energy efficiency.

desk verdict A sensible hybrid SLC/MLC RRAM PIM idea undone, as written, by an unspecified SLC-vs-MLC noise differential that the accuracy curves depend on. read the letter →

arxiv 2506.00020 v1 pith:ZSMQA3PB submitted 2025-05-20 cs.AR

Chang Eun Song , Priyansh Bhatnagar , Zihan Xia , Nam Sung Kim , Tajana Rosing , Mingu Kang This is my paper · ORCID
classification cs.AR
keywords processing-in-memoryRRAMtransformeraccelerationsingularvaluedecompositiongradientredistributionSLC-MLChybridmemorymixed-signalPIManalog
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a hybrid resistive random-access memory (RRAM) processing-in-memory (PIM) accelerator can run Transformers far more efficiently without the accuracy collapse that analog multi-level-cell (MLC) memory usually causes. The proposed HyFlexPIM stores static weights in two tiers: a small number of critical weights in single-level-cell (SLC) RRAM, which is less noisy and costlier, and the remaining roughly 90-95% in 2-bit MLC RRAM, which is denser and faster but error-prone. The enabling software move is gradient redistribution: Singular Value Decomposition (SVD), hard-threshold truncation, and 1-3 epochs of fine-tuning that concentrate loss sensitivity into a few top singular values. With this, only 5-10% of encoder and vision weights and 5-20% of decoder weights need SLC protection to keep accuracy loss under 1% and decoder loss increase under 10%, while throughput rises up to 1.86x and energy efficiency up to 1.45x versus state-of-the-art. The broader point is that instead of passively relying on the inherent error-resilience of neural networks, one can deliberately reshape the model so that the hardware protects only a tiny fraction of it.

What carries the argument

The load-bearing mechanism is SVD-based gradient redistribution, a two-step transformation of every static weight matrix. First, $W = U\Sigma V^\top$ is truncated at a hard threshold $D_{\mathrm{Th}} = D_{h1}D_{h2}/(D_{h1}+D_{h2})$ so the number of parameters and MACs after merging $\Sigma V^\top$ is unchanged; then fine-tuning concentrates the gradient $\partial\mathcal{L}/\partial w_i$ onto the leading singular values, creating a clear importance ranking. On the hardware side, the matching object is a reconfigurable analog PIM module whose 64x128 RRAM arrays can be read as 1-bit SLC or 2-bit MLC cells, with a flexible SAR ADC that bypasses its MSB capacitor to switch between 6-bit and 7-bit conversion at under 1% area and energy overhead. The gradient magnitude of each singular value selects which weights go to SLC and which to MLC; this gradient-based selection, not raw weight magnitude or singular-value order alone, is what the paper argues preserves accuracy at 5-20% SLC rates.

What would settle it

Repeat the accuracy sweep while assigning SLC cells the same Gaussian noise standard deviation as the MLC cells, namely the value reverse-engineered from a 4.04% bit-error rate; if BERT-Base GLUE accuracy then drops by more than 1% at a 5% SLC rate, the paper's claim that placing 5-10% of weights in SLC is what preserves accuracy would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the apparent trade-off between analog RRAM efficiency and Transformer accuracy can be broken by co-designing the model and the memory. After applying SVD to each static weight matrix and truncating at a hard threshold that keeps the MAC count the same as the original, the authors fine-tune for 1-3 epochs. That fine-tuning does double duty: it recovers the accuracy lost to truncation and it redistributes gradients so that the top singular values carry nearly all of the loss sensitivity. HyFlexPIM then maps the weights corresponding to those top-k% singular values into SLC RRAM and everything else into 2-bit MLC RRAM. On BERT-Base, BERT-Large, ViT-Base, GPT-2, and Llama3, the reported result is less than 1% accuracy drop for encoder and vision models, less than 10% loss increase for decoder models, and up to 1.86x throughput and 1.45x energy efficiency relative to the best SLC-only hybrid baseline and other state-of-the-art systems.

Load-bearing premise

The accuracy results assume that SLC RRAM cells are substantially less noisy than 2-bit MLC cells, but the simulator injects a single Gaussian noise level calibrated to a 4.04% MLC bit-error rate into all linear layers and does not specify a separate, lower SLC noise level; if SLC noise is not much lower, protecting only 5-10% of weights in SLC will not preserve accuracy.

Editorial extensions

If this is right

  • Encoder and vision transformers keep accuracy within 1% of an INT8 baseline when only 5-10% of weights are stored in SLC, with the rest running in 2-bit MLC.
  • Decoder-style models such as GPT-2 and Llama3 keep loss increase below 10% at a 20% SLC rate, so autoregressive generation can use the same hybrid mapping.
  • Because 2-bit MLC packs two weight bits per cell, the same array and ADC energy deliver roughly double the linear-layer throughput and about half the analog computation energy per useful bit.
  • The single reconfigurable analog module switches between SLC and MLC with under 1% area and energy overhead, letting one fixed chip adapt the SLC/MLC split per model without a second memory technology.
  • SVD truncation at the hard threshold keeps the parameter count and MAC count the same as the original dense matrix, so the accuracy and efficiency gains are not bought with extra computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gradient-redistribution recipe is transportable: any noisy storage technology with two reliability tiers, such as PCM or DRAM refresh tiers, could use the same SVD-plus-fine-tuning trick to minimize use of the costly, reliable tier.
  • The paper's protection-rate sweep suggests that an adaptive per-layer SLC allocation might beat one global k%, since harder GLUE tasks needed 10-30% SLC while easier ones needed only 5%; this is an extension the paper does not explore.
  • If SLC and MLC noise levels are later measured separately on a real chip, the reported 5-10% SLC rates would need to be re-derived, because the current simulation couples both tiers to a single noise parameter calibrated to an MLC bit-error rate.
  • The reshape-then-protect logic could also combine with attention token pruning or structured sparsity, since the offline SVD and fine-tuning do not touch the dynamic attention path.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HyFlexPIM, a processing-in-memory accelerator for Transformer inference that combines digital RRAM PIM for attention and non-linear operations with analog RRAM PIM for linear layers. The analog PIM is reconfigurable between SLC and 2-bit MLC operation, and a software technique called 'gradient redistribution' uses SVD truncation plus fine-tuning to concentrate weight importance into a small fraction of entries, which are then mapped to SLC while the rest use MLC. The manuscript claims that protecting only 5-10% of weights (encoder/vision) or 5-20% (decoder) in SLC keeps accuracy degradation below 1% (or loss increase below 10%), while delivering up to 1.86x throughput and 1.45x energy efficiency over state-of-the-art baselines.

Significance. If the claims hold, the paper would be a useful contribution to mixed-signal PIM for Transformers: it proposes a concrete reconfigurable SLC/MLC architecture, a gradient-based importance metric for hybrid mapping, and a hardware-algorithm co-design that goes beyond reliance on inherent model resiliency. The evaluation is broad (BERT, GPT-2, Llama3, ViT) and uses device-derived noise data from fabricated RRAM chips, which is a strength. The simulator, however, is not shipped and, more importantly, the noise model lacks a separate SLC noise specification, which makes the central accuracy result unreproducible as written. The reported SLC rates and energy ratios also contain internal inconsistencies that need correction.

major comments (3)
  1. [Section 5.2, Eq. (5)] The noise model applies a single Gaussian perturbation, calibrated to a 4.04% MLC bit-error rate, to every linear layer, with no separate noise parameter for SLC cells. The central device-level premise (Section 3.2) is that SLC has a much higher noise margin than MLC, and the entire accuracy argument of Figure 12 rests on protecting a small fraction of weights in SLC. As written, the simulator injects the same MLC-level noise into SLC-mapped weights, so the reported accuracy recovery as the SLC rate increases is not reproducible from the described model. Please specify the SLC noise model (e.g., noise-free, a distinct sigma, or a device-derived BER) and justify it with the cited measurements; otherwise the accuracy results are unverifiable.
  2. [Section 6.1, Figure 12; Abstract and Section 1] The claims that only 5-10% of encoder weights and 5-20% of decoder weights need SLC protection are not supported by the per-task results. In Figure 12, BERT-Base CoLA, QQP, SST-2, and RTE require SLC rates of 10-30% to stay within 1% of the baseline; BERT-Large also requires 10-30% for several tasks; and GPT-2 and Llama3 require 20% to keep loss increase below 10%. The evaluation uses an unconstrained per-task selection of the SLC rate, while the abstract and contributions present 5-10% (or 5-20%) as a general property. Please report the actual SLC rate needed for every task/model and clarify whether the efficiency numbers use those task-specific rates or a fixed budget.
  3. [Abstract, Section 1, Section 6.3.1] The reported maximum energy efficiency is internally inconsistent. The abstract and Section 2 mention 1.45x; the contributions in Section 1 mention 1.24x; Section 6.3.1 reports 1.44x for BERT-Large and 1.67x for GPT-2 end-to-end against ASADI†; and Figure 15 labels energy-efficiency ratios of 1.45x and 1.44x. Please reconcile these numbers and state the exact baseline, configuration, and SLC rate for each ratio.
minor comments (5)
  1. [Section 1] The acronym 'SPIRNT' is a typo for 'SPRINT'.
  2. [Section 6 heading] The heading 'Comparsion' should be 'Comparison'.
  3. [Figure 12 caption] The caption uses 'Baseline' and 'FP w/o SVD' while the text uses 'FP w/o SVD'; please make the labels consistent and define 'G-AVG' in the caption.
  4. [Section 5.2] The text says the Gaussian standard deviation was 'reverse-calculated' from the 4.04% BER but does not report the resulting sigma value or whether the same sigma was used for all model sizes and layers; please include this value and any dependence on cell type.
  5. [Section 3.1] The text says HyFlexPIM includes 24 PUs for 24 layers but later notes BERT-Base and GPT-2 have 12 layers and benefit from 2x throughput; please clarify the default mapping and when tensor parallelism is invoked.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the 5–10% SLC claim is an empirical, measured outcome of gradient redistribution, and the efficiency numbers are benchmarked against external baselines.

full rationale

The central claims are not circular. Gradient redistribution (Section 4) is an empirical algorithm: SVD truncation with a hard threshold chosen so DTh = Dh1·Dh2/(Dh1+Dh2) preserves parameter and MAC count by construction, and fine-tuning concentrates gradients; the 5–10% SLC claim is then measured, not assumed. Accuracy is evaluated by injecting noise from measured RRAM cells (Eq. 5) and comparing SLC rates against an INT8 no-noise baseline and external FP baselines (Figure 12); efficiency is benchmarked against ASADI, SPRINT, TransPIM/NMP, and non-PIM baselines with scaled 65 nm components. The only overlap with prior work by the same group is the RRAM BER source [15] (co-author T. Rosing) and baseline implementations [75,79] (co-authors M. Kang and T. Rosing), but those are used as measured external inputs or comparison points, not as hidden assumptions that force the result. The under-specification in Section 5.2, where Eq. 5 applies one MLC-calibrated Gaussian to all linear layers without stating a separate SLC noise level, would make the reported SLC-rate-dependent recovery curves hard to reproduce from the manuscript as written; that is a reproducibility and modeling gap, not a reduction of the predicted quantity to a fitted input. No equation or definition in the paper makes the accuracy gain from SLC mapping equal to its own input by construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three main underpinnings: a per-task chosen SLC protection rate, a calibrated Gaussian noise model whose SLC-specific behavior is unspecified, and the gradient-magnitude proxy for noise sensitivity. There are no invented physical entities; all components are standard RRAM, ADC, and S&A designs.

free parameters (1)
  • SLC protection rate k = 5%, 10%, 20%, 30% per task/model
    Chosen per task (Figure 12) to keep accuracy drop below 1% or loss increase below 10%; not predicted a priori, and some GLUE tasks require 10-30%, which conflicts with the abstract's 5-10% claim.
assumptions (4)
  • domain assumption SVD truncation at rank D_Th = (D_h1 * D_h2) / (D_h1 + D_h2) preserves the parameter count and computational cost, and subsequent fine-tuning for 1-3 epochs recovers the pre-SVD accuracy.
    Used in Section 4.1; no proof that 1-3 epochs is sufficient across all tasks, but empirically claimed for the evaluated benchmarks.
  • domain assumption Gaussian multiplicative noise in Eq. 5 with a standard deviation calibrated to match the MLC bit-error rate reported in [15] captures the accuracy-relevant non-idealities of both SLC and MLC RRAM.
    Load-bearing for all accuracy results; the paper does not specify separate SLC noise parameters, so this assumption is unstated and critical (Section 5.2).
  • domain assumption The magnitude of the loss gradient with respect to each singular value is a valid proxy for how much accuracy is lost if that rank is stored in noisier MLC cells.
    This is the conceptual core of gradient redistribution (Section 4.2). If loss gradients do not track noise sensitivity, the SLC/MLC mapping strategy fails.
  • standard math Energy and area estimates from NVSIM, [31], [52], and other cited sources, scaled to 65nm, are representative of a real implementation.
    All hardware efficiency numbers rest on these models (Section 5.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution." pith.science (2026). https://pith.science/paper/ZSMQA3PB

@misc{pith2026250600020,
  author       = {Pith},
  title        = {Pith review of: Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSMQA3PB}},
  note         = {Machine review of arXiv:2506.00020}
}
read the original abstract

Transformers, while revolutionary, face challenges due to their demanding computational cost and large data movement. To address this, we propose HyFlexPIM, a novel mixed-signal processing-in-memory (PIM) accelerator for inference that flexibly utilizes both single-level cell (SLC) and multi-level cell (MLC) RRAM technologies to trade-off accuracy and efficiency. HyFlexPIM achieves efficient dual-mode operation by utilizing digital PIM for high-precision and write-intensive operations while analog PIM for high parallel and low-precision computations. The analog PIM further distributes tasks between SLC and MLC PIM operations, where a single analog PIM module can be reconfigured to switch between two operations (SLC/MLC) with minimal overhead (<1% for area & energy). Critical weights are allocated to SLC RRAM for high accuracy, while less critical weights are assigned to MLC RRAM to maximize capacity, power, and latency efficiency. However, despite employing such a hybrid mechanism, brute-force mapping on hardware fails to deliver significant benefits due to the limited proportion of weights accelerated by the MLC and the noticeable degradation in accuracy. To maximize the potential of our hybrid hardware architecture, we propose an algorithm co-optimization technique, called gradient redistribution, which uses Singular Value Decomposition (SVD) to decompose and truncate matrices based on their importance, then fine-tune them to concentrate significance into a small subset of weights. By doing so, only 5-10% of the weights have dominantly large gradients, making it favorable for HyFlexPIM by minimizing the use of expensive SLC RRAM while maximizing the efficient MLC RRAM. Our evaluation shows that HyFlexPIM significantly enhances computational throughput and energy efficiency, achieving maximum 1.86X and 1.45X higher than state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2506.00020 by the authors.

Figure 2
Figure 2. The number of operations for each stage in trans [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of (a) analog RRAM processing-in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Post-SVD matrix applied to static weights in the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (12 more)
Figure 5
Figure 5. Figure 5: Architecture overview of the proposed HyFlexPIM, which is based on a scalable architecture with hybrid analog and [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: SLC PIM mapping and Shift & Adder operation. [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: (a) 6 and 7 bits flexible SAR-ADC schematic, and (b) [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: The proposed mapping of transformer operations [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 10
Figure 10. Figure 10: Hard-Thresholding and Fine-tuning the post-SVD [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Gradient distribution change for FC layer in BERT-Base: (a) gradient of each weight (order-wise) in a row before [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Accuracy trends with respect to the SLC rate for (a) BERT-Base, BERT-Large, (b) GPT-2, Llama3, and ViT. [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Accuracy comparison between weight magnitude [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: Normalized energy consumption of linear layers compared to the energy of ASADI [PITH_FULL_IMAGE:figures/full_fig_p011_14.png]
Figure 15
Figure 15. Figure 15: End-to-end energy: (a) comparison with SOA, and (b) breakdown with HyFlexPIM at 5% SLC for BERT-Large, and (c) [PITH_FULL_IMAGE:figures/full_fig_p011_15.png]
Figure 16
Figure 16. Figure 16: Speedup factor compared to ASADI† [31] and SPRINT [75] with respect to the SLC rate and the sequence length. better selection of critical parameters, ultimately leading to higher accuracy and robustness. 6.3 Comparsion with Modern Accelerators We evaluate HyFlexPIM ag…
Figure 17
Figure 17. Figure 17: This results in only a minor degradation compared to the [PITH_FULL_IMAGE:figures/full_fig_p012_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 67 canonical work pages

  1. [1]

    Singular value decomposition (svd) and generalized singular value decomposition

    Hervé Abdi. Singular value decomposition (svd) and generalized singular value decomposition. Encyclopedia of measurement and statistics , 907(912):44, 2007

    work page 2007

  2. [2]

    Xcel-ram: Accelerating binary neural networks in high-throughput sram compute arrays

    Amogh Agrawal, Akhilesh Jaiswal, Deboleena Roy, Bing Han, Gopalakrishnan Srinivasan, Aayush Ankit, and Kaushik Roy. Xcel-ram: Accelerating binary neural networks in high-throughput sram compute arrays. IEEE Transactions on Circuits and Systems I: Regular Papers , 66(8):3064–3076, 2019

    work page 2019

  3. [3]

    Llama 3.2: Multilingual large language models

    Meta AI. Llama 3.2: Multilingual large language models. https://www.llama.com, 2024

    work page 2024

  4. [4]

    In-memory low-cost bit-serial addition using commodity dram technology

    Mustafa F Ali, Akhilesh Jaiswal, and Kaushik Roy. In-memory low-cost bit-serial addition using commodity dram technology. IEEE Transactions on Circuits and Systems I: Regular Papers , 67(1):155–165, 2019

    work page 2019

  5. [5]

    Tanner Andrulis, Joel S Emer, and Vivienne Sze. Raella: Reforming the arithmetic for efficient, low-resolution, and low-loss analog pim: No retraining required! In Proceedings of the 50th Annual International Symposium on Computer Architecture, pages 1–16, 2023

    work page 2023

  6. [6]

    Artisan Memory Compilers

    ARM. Artisan Memory Compilers. https://developer.arm.com/ip-products/ physical-ip/embedded-memory, 2021. Accessed: 2021-11-08

    work page 2021

  7. [7]

    Longformer: The long- document transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long- document transformer. arXiv preprint arXiv:2004.05150, 2020

    arXiv 2004

  8. [8]

    Improving language models by retrieving from trillions of tokens

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Ruther- ford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bog- dan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning , pages 2206–

    work page

Show all 79 references
  1. [9]

    Neural-pim: Efficient processing-in-memory with neural approximation of pe- ripherals

    Weidong Cao, Yilong Zhao, Adith Boloor, Yinhe Han, Xuan Zhang, and Li Jiang. Neural-pim: Efficient processing-in-memory with neural approximation of pe- ripherals. IEEE Transactions on Computers , 71(9):2142–2155, 2021

    2021

  2. [10]

    The devil is in details: Delving into lite ffn design for vision transformers

    Zhiyang Chen, Yousong Zhu, Zhaowen Li, Fan Yang, Chaoyang Zhao, Jinqiao Wang, and Ming Tang. The devil is in details: Delving into lite ffn design for vision transformers. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ,...

    2024

  3. [11]

    Prime: a novel processing-in-memory architecture for neural network computation in reram-based main memory

    Ping Chi, Shuangchen Li, Cong Xu, Tao Zhang, Jishen Zhao, Yongpan Liu, Yu Wang, and Yuan Xie. Prime: a novel processing-in-memory architecture for neural network computation in reram-based main memory. In Proceedings of the 43rd International Symposium on Computer Architecture...

    2016

  4. [12]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  5. [13]

    Xiangyu Dong, Cong Xu, Yuan Xie, and Norman P. Jouppi. Nvsim: A circuit- level performance, energy, and area model for emerging nonvolatile memory. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 31(7):994–1007, 2012

    2012

  6. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  7. [15]

    Keming Fan, Wei-Chen Chen, Sumukh Pinge, H. S. Philip Wong, and Tajana Ros- ing. Efficient open modification spectral library searching in high-dimensional space with multi-level-cell memory, 2024

    2024

  8. [16]

    Computedram: In-memory compute using off-the-shelf drams

    Fei Gao, Georgios Tziantzioulis, and David Wentzlaff. Computedram: In-memory compute using off-the-shelf drams. In Proceedings of the 52nd annual IEEE/ACM international symposium on microarchitecture , page 100–113, 2019

    2019

  9. [17]

    A variation-tolerant in-memory machine learning classifier via on-chip training

    Sujan K Gonugondla, Mingu Kang, and Naresh R Shanbhag. A variation-tolerant in-memory machine learning classifier via on-chip training. JSSC, 53(11):3163– 3173, November 2018

    2018

  10. [18]

    Sabry, Marios Barlas, Laurent Grenouillet, Jean Coignus, Edith Beigne, Tony Wu, Binh Q

    Alessandro Grossi, Elisa Vianello, Mohamed M. Sabry, Marios Barlas, Laurent Grenouillet, Jean Coignus, Edith Beigne, Tony Wu, Binh Q. Le, Mary K. Wootters, Cristian Zambelli, Etienne Nowak, and Subhasish Mitra. Resistive ram endurance: Array-level characterization and correcti...

    2019

  11. [19]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  12. [20]

    A nonvolatile al-edge processor with 4mb slc-mlc hybrid-mode reram compute-in-memory macro and 51.4-251tops/w

    Wei-Hsing Huang, Tai-Hao Wen, Je-Min Hung, Win-San Khwa, Yun-Chen Lo, Chuan-Jia Jhang, Huna-Hsi Hsu, Yu-Hsiana Chin, Yu-Chiao Chen, Chuna-Chuan Lo, et al. A nonvolatile al-edge processor with 4mb slc-mlc hybrid-mode reram compute-in-memory macro and 51.4-251tops/w. In 2023 IEE...

    2023

  13. [21]

    A four-megabit compute-in-memory macro with eight-bit precision based on cmos and resistive random-access memory for ai edge devices.Nature Electronics, 4(12):921–930, 2021

    Je-Min Hung, Cheng-Xin Xue, Hui-Yao Kao, Yen-Hsiang Huang, Fu-Chun Chang, Sheng-Po Huang, Ta-Wei Liu, Chuan-Jia Jhang, Chin-I Su, Win-San Khwa, et al. A four-megabit compute-in-memory macro with eight-bit precision based on cmos and resistive random-access memory for ai edge d...

    2021

  14. [22]

    Floatpim: In- memory acceleration of deep neural network training with high precision

    Mohsen Imani, Saransh Gupta, Yeseong Kim, and Tajana Rosing. Floatpim: In- memory acceleration of deep neural network training with high precision. In Proceedings of the 46th International Symposium on Computer Architecture , pages 802–815, 2019

    2019

  15. [23]

    Circuit design of rram-based neuromorphic hardware systems for classification and modified hebbian learning

    Yuning Jiang, Peng Huang, Zheng Zhou, and Jinfeng Kang. Circuit design of rram-based neuromorphic hardware systems for classification and modified hebbian learning. Science China Information Sciences , 62:1–19, 2019

    2019

  16. [24]

    Rehy: A reram-based digital/analog hybrid pim architecture for accelerating cnn training

    Hai Jin, Cong Liu, Haikun Liu, Ruikun Luo, Jiahong Xu, Fubing Mao, and Xi- aofei Liao. Rehy: A reram-based digital/analog hybrid pim architecture for accelerating cnn training. IEEE Transactions on Parallel and Distributed Systems , 33(11):2872–2884, 2021

    2021

  17. [25]

    Rehy: A reram-based digital/analog hybrid pim architecture for accelerating cnn training

    Hai Jin, Cong Liu, Haikun Liu, Ruikun Luo, Jiahong Xu, Fubing Mao, and Xiaofei Liao. Rehy: A reram-based digital/analog hybrid pim architecture for accelerating cnn training. IEEE Transactions on Parallel and Distributed Systems , 33(11):2872– 2884, 2022

    2022

  18. [26]

    Ten lessons from three generations shaped google’s tpuv4i: Industrial product

    Norman P Jouppi, Doe Hyun Yoon, Matthew Ashcraft, Mark Gottscho, Thomas B Jablin, George Kurian, James Laudon, Sheng Li, Peter Ma, Xiaoyu Ma, et al. Ten lessons from three generations shaped google’s tpuv4i: Industrial product. In Chang Eun Song, Priyansh Bhatnagar, Zihan Xia,...

    2021

  19. [27]

    Samba: spar- sity aware in-memory computing based machine learning accelerator

    Dong Eun Kim, Aayush Ankit, Cheng Wang, and Kaushik Roy. Samba: spar- sity aware in-memory computing based machine learning accelerator. IEEE Transactions on Computers, 72(9):2615–2627, 2023

    2023

  20. [28]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009

    2009

  21. [29]

    A 3.1 mw 8b 1.2 gs/s single-channel asynchronous sar adc with alternate comparators for enhanced speed in 32 nm digital soi cmos

    Lukas Kull, Thomas Toifl, Martin Schmatz, Pier Andrea Francese, Christian Menolfi, Matthias Braendli, Marcel Kossel, Thomas Morf, Toke Meyer Andersen, and Yusuf Leblebici. A 3.1 mw 8b 1.2 gs/s single-channel asynchronous sar adc with alternate comparators for enhanced speed in...

    2013

  22. [30]

    25.4 a 20nm 6gb function-in-memory dram, based on hbm2 with a 1.2 tflops pro- grammable computing unit using bank-level parallelism, for machine learning applications

    Young-Cheon Kwon, Suk Han Lee, Jaehoon Lee, Sang-Hyuk Kwon, Je Min Ryu, Jong-Pil Son, O Seongil, Hak-Soo Yu, Haesuk Lee, Soo Young Kim, et al. 25.4 a 20nm 6gb function-in-memory dram, based on hbm2 with a 1.2 tflops pro- grammable computing unit using bank-level parallelism, f...

    2021

  23. [31]

    Asadi: Accelerating sparse attention using diagonal-based in-situ computing

    Huize Li, Zhaoying Li, Zhenyu Bai, and Tulika Mitra. Asadi: Accelerating sparse attention using diagonal-based in-situ computing. In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , pages 774–787. IEEE, 2024

    2024

  24. [32]

    Hardsea: Hybrid analog-reram clustering and digital-sram in-memory computing accelerator for dynamic sparse self- attention in transformer

    Shiwei Liu, Chen Mu, Hao Jiang, Yunzhengmao Wang, Jinshan Zhang, Feng Lin, Keji Zhou, Qi Liu, and Chixiao Chen. Hardsea: Hybrid analog-reram clustering and digital-sram in-memory computing accelerator for dynamic sparse self- attention in transformer. IEEE Transactions on Very...

    2023

  25. [33]

    An area-and energy-efficient spiking neural network with spike-time-dependent plasticity realized with sram processing-in-memory macro and on-chip unsupervised learning

    Shuang Liu, JJ Wang, JT Zhou, SG Hu, Qi Yu, TP Chen, and Yang Liu. An area-and energy-efficient spiking neural network with spike-time-dependent plasticity realized with sram processing-in-memory macro and on-chip unsupervised learning. IEEE Transactions on Biomedical Circuits...

    2023

  26. [34]

    Hard- ware accelerator for multi-head attention and position-wise feed-forward in the transformer

    Siyuan Lu, Meiqi Wang, Shuang Liang, Jun Lin, and Zhongfeng Wang. Hard- ware accelerator for multi-head attention and position-wise feed-forward in the transformer. In 2020 IEEE 33rd International System-on-Chip Conference (SOCC) , pages 84–89. IEEE, 2020

    2020

  27. [35]

    Lightformer: Light-weight transformer using svd-based weight transfer and parameter sharing

    Xiuqing Lv, Peng Zhang, Sunzhu Li, Guobing Gan, and Yueheng Sun. Lightformer: Light-weight transformer using svd-based weight transfer and parameter sharing. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 10323–10335, 2023

    2023

  28. [36]

    Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz

    Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank.Computational Linguistics, 19(2):313–330, 1993

    1993

  29. [37]

    The WikiText Long Term Dependency Language Model- ing Dataset

    Stephen Merity. The WikiText Long Term Dependency Language Model- ing Dataset. https://blog.salesforceairesearch.com/the-wikitext-long-term- dependency-language-modeling-dataset/, 2021. Accessed: 2021-11-08

    2021

  30. [38]

    Pointer sentinel mixture models, 2016

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016

    2016

  31. [39]

    Leave no con- text behind: Efficient infinite context transformers with infini-attention

    Tsendsuren Munkhdalai, Manaal Faruqui, and Siddharth Gopal. Leave no con- text behind: Efficient infinite context transformers with infini-attention. arXiv preprint arXiv:2404.07143, 2024

    2024 arXiv

  32. [40]

    ADC Performance Survey 1997-2024

    Boris Murmann. ADC Performance Survey 1997-2024. [Online]. Available: https://github.com/bmurmann/ADC-survey

    1997

  33. [41]

    Fine-grained dram: Energy- efficient dram for extreme bandwidth systems

    Mike O’Connor, Niladrish Chatterjee, Donghyuk Lee, John Wilson, Aditya Agrawal, Stephen W Keckler, and William J Dally. Fine-grained dram: Energy- efficient dram for extreme bandwidth systems. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitect...

    2017

  34. [42]

    Training and operation of an integrated neuromorphic network based on metal-oxide memristors

    Mirko Prezioso, Farnood Merrikh-Bayat, Brian D Hoskins, Gina C Adam, Kon- stantin K Likharev, and Dmitri B Strukov. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 521(7550):61– 64, 2015

    2015

  35. [43]

    Fact: Ffn-attention co-optimized transformer architecture with eager correlation prediction

    Yubin Qin, Yang Wang, Dazheng Deng, Zhiren Zhao, Xiaolong Yang, Leibo Liu, Shaojun Wei, Yang Hu, and Shouyi Yin. Fact: Ffn-attention co-optimized transformer architecture with eager correlation prediction. In Proceedings of the 50th Annual International Symposium on Computer A...

    2023

  36. [44]

    Radford, Jeffrey Wu, R

    A. Radford, Jeffrey Wu, R. Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019

    2019

  37. [45]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  38. [47]

    Adap- tive programming in multi-level cell reram

    Misbah Ramadan, Nicolás Wainstein, Ran Ginosar, and Shahar Kvatinsky. Adap- tive programming in multi-level cell reram. Microelectronics Journal, 90:169–180, 2019

    2019

  39. [48]

    Associative learning of a three-terminal memristor network for digits recognition

    Yiming Ren, Bobo Tian, Mengge Yan, Guangdi Feng, Bin Gao, Fangyu Yue, Hui Peng, Xiaodong Tang, Qiuxiang Zhu, Junhao Chu, et al. Associative learning of a three-terminal memristor network for digits recognition. Science China Information Sciences, 66(2):122403, 2023

    2023

  40. [49]

    Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation adcs

    Mehdi Saberi, Reza Lotfi, Khalil Mafinezhad, and Wouter A Serdijn. Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation adcs. IEEE Transactions on Circuits and Systems I: Regular Papers, 58(8):1736–1748, 2011

    2011

  41. [50]

    Stanley Williams, and Vivek Srikumar

    Ali Shafiee, Anirban Nag, Naveen Muralimanohar, Rajeev Balasubramonian, John Paul Strachan, Miao Hu, R. Stanley Williams, and Vivek Srikumar. Isaac: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In Proceedings of the 43rd International...

    2016

  42. [51]

    Stanley Williams, and Vivek Srikumar

    Ali Shafiee, Anirban Nag, Naveen Muralimanohar, Rajeev Balasubramonian, John Paul Strachan, Miao Hu, R. Stanley Williams, and Vivek Srikumar. Isaac: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. SIGARCH Comput. Archit. News, 44(3):14–2...

    2016

  43. [52]

    Isaac: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars

    Ali Shafiee, Anirban Nag, Naveen Muralimanohar, Rajeev Balasubramonian, John Paul Strachan, Miao Hu, R Stanley Williams, and Vivek Srikumar. Isaac: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ACM SIGARCH Computer Architecture News, 4...

    2016

  44. [53]

    Pci express ® 6.0 specification at 64.0 gt/s with pam- 4 signaling: a low latency, high bandwidth, high reliability and cost-effective interconnect

    Debendra Das Sharma. Pci express ® 6.0 specification at 64.0 gt/s with pam- 4 signaling: a low latency, high bandwidth, high reliability and cost-effective interconnect. In 2020 IEEE Symposium on High-Performance Interconnects (HOTI) , pages 1–8. IEEE, 2020

    2020

  45. [54]

    Relaxing non-volatility for fast and energy-efficient stt-ram caches

    Clinton W Smullen, Vidyabhushan Mohan, Anurag Nigam, Sudhanva Guru- murthi, and Mircea R Stan. Relaxing non-volatility for fast and energy-efficient stt-ram caches. In 2011 IEEE 17th International Symposium on High Performance Computer Architecture, pages 50–61. IEEE, 2011

    2011

  46. [55]

    52.5 tops/w 1.7 ghz reconfigurable xgboost inference accelerator based on modular-unit-tree with dynamic data and compute gating

    Chang Eun Song, Yidong Li, Amardeep Ramnani, Pulkit Agrawal, Purvi Agrawal, Sung-Joon Jang, Sang-Seol Lee, Tajana Rosing, and Mingu Kang. 52.5 tops/w 1.7 ghz reconfigurable xgboost inference accelerator based on modular-unit-tree with dynamic data and compute gating. In 2024 I...

    2024

  47. [56]

    Ef- ficient transformer acceleration via reconfiguration for encoder and decoder mod- els and sparsity-aware algorithm mapping

    Chang Eun Song, Ashkan Moradifirouzabadi, Tajana Rosing, and Mingu Kang. Ef- ficient transformer acceleration via reconfiguration for encoder and decoder mod- els and sparsity-aware algorithm mapping. In Proceedings of the 29th ACM/IEEE International Symposium on Low Power Ele...

    2024

  48. [57]

    Scaling equations for the accurate prediction of cmos device performance from 180 nm to 7 nm

    Aaron Stillmaker and Bevan Baas. Scaling equations for the accurate prediction of cmos device performance from 180 nm to 7 nm. Integration, 58:74–81, 2017

    2017

  49. [58]

    Logic de- sign within memristive memories using memristor-aided logic (magic)

    Nishil Talati, Saransh Gupta, Pravin Mane, and Shahar Kvatinsky. Logic de- sign within memristive memories using memristor-aided logic (magic). IEEE Transactions on Nanotechnology, 15(4):635–650, 2016

    2016

  50. [59]

    Exploring transformers in natural language generation: Gpt, bert, and xlnet

    M Onat Topal, Anil Bas, and Imke van Heerden. Exploring transformers in natural language generation: Gpt, bert, and xlnet. arXiv preprint arXiv:2102.08036, 2021

    2021 arXiv

  51. [60]

    Racer: Bit-pipelined process- ing using resistive memory

    Minh SQ Truong, Eric Chen, Deanyone Su, Liting Shen, Alexander Glass, L Richard Carley, James A Bain, and Saugata Ghose. Racer: Bit-pipelined process- ing using resistive memory. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, page 100–116, 2021

    2021

  52. [61]

    33.1 a 74 tmacs/w cmos-rram neurosynaptic core with dynamically reconfigurable dataflow and in-situ transposable weights for probabilistic graphical models

    Weier Wan, Rajkumar Kubendran, S Burc Eryilmaz, Wenqiang Zhang, Yan Liao, Dabin Wu, Stephen Deiss, Bin Gao, Priyanka Raina, Siddharth Joshi, et al. 33.1 a 74 tmacs/w cmos-rram neurosynaptic core with dynamically reconfigurable dataflow and in-situ transposable weights for prob...

    2020

  53. [62]

    Glue: A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018

    2018 arXiv

  54. [63]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding, 2019

    2019

  55. [64]

    Hat: Hardware-aware transformers for efficient natural language processing

    Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, and Song Han. Hat: Hardware-aware transformers for efficient natural language processing. arXiv preprint arXiv:2005.14187, 2020

    2005 arXiv

  56. [65]

    Augmenting language models with long-term memory

    Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. Augmenting language models with long-term memory. Advances in Neural Information Processing Systems , 36, 2024

    2024

  57. [66]

    Beacongnn: Large-scale gnn acceleration with out-of-order streaming in-storage computing

    Yuyue Wang, Xiurui Pan, Yuda An, Jie Zhang, and Glenn Reinman. Beacongnn: Large-scale gnn acceleration with out-of-order streaming in-storage computing. In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 330–344. IEEE, 2024

    2024

  58. [67]

    Phase change memory

    H-S Philip Wong, Simone Raoux, SangBum Kim, Jiale Liang, John P Reifenberg, Bipin Rajendran, Mehdi Asheghi, and Kenneth E Goodson. Phase change memory. Proceedings of the IEEE , 98(12):2201–2227, 2010

    2010

  59. [68]

    Pim gpt a hybrid process in memory accelerator for autoregressive transformers

    Yuting Wu, Ziyu Wang, and Wei D Lu. Pim gpt a hybrid process in memory accelerator for autoregressive transformers. npj Unconventional Computing , 1(1):4, 2024. Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution

    2024

  60. [69]

    16.1 a 22nm 4mb 8b-precision reram computing-in-memory macro with 11.91 to 195.7 tops/w for tiny ai edge devices

    Cheng-Xin Xue, Je-Min Hung, Hui-Yao Kao, Yen-Hsiang Huang, Sheng-Po Huang, Fu-Chun Chang, Peng Chen, Ta-Wei Liu, Chuan-Jia Jhang, Chin-I Su, et al. 16.1 a 22nm 4mb 8b-precision reram computing-in-memory macro with 11.91 to 195.7 tops/w for tiny ai edge devices. In 2021 IEEE In...

    2021

  61. [70]

    Processing-in-memory using optically-addressed phase change memory

    Guowei Yang, Cansu Demirkiran, Zeynep Ece Kizilates, Carlos A Ríos Ocampo, Ayse K Coskun, and Ajay Joshi. Processing-in-memory using optically-addressed phase change memory. In 2023 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) , pages 1–6. IEEE, 2023

    2023

  62. [71]

    Fsl-hdnn: A 5.7 tops/w end-to-end few-shot learning classifier accelerator with feature extraction and hyperdimensional computing

    Haichao Yang, Chang Eun Song, Weihong Xu, Behnam Khaleghi, Uday Mallappa, Monil Shah, Keming Fan, Mingu Kang, and Tajana Rosing. Fsl-hdnn: A 5.7 tops/w end-to-end few-shot learning classifier accelerator with feature extraction and hyperdimensional computing. In 2024 IEEE Euro...

    2024

  63. [72]

    Retransformer: Reram-based processing-in-memory architecture for transformer acceleration

    Xiaoxuan Yang, Bonan Yan, Hai Li, and Yiran Chen. Retransformer: Reram-based processing-in-memory architecture for transformer acceleration. In Proceedings of the 39th International Conference on Computer-Aided Design , pages 1–9, 2020

    2020

  64. [73]

    Retransformer: Reram-based processing-in-memory architecture for transformer acceleration

    Xiaoxuan Yang, Bonan Yan, Hai Li, and Yiran Chen. Retransformer: Reram-based processing-in-memory architecture for transformer acceleration. In Proceedings of the 39th International Conference on Computer-Aided Design , ICCAD ’20, New York, NY, USA, 2020. Association for Compu...

    2020

  65. [74]

    Fully hardware-implemented memristor convolutional neural network

    Peng Yao, Huaqiang Wu, Bin Gao, Jianshi Tang, Qingtian Zhang, Wenqiang Zhang, J Joshua Yang, and He Qian. Fully hardware-implemented memristor convolutional neural network. Nature, 577(7792):641–646, 2020

    2020

  66. [75]

    Sparse attention acceleration with synergistic in-memory pruning and on-chip recomputation

    Amir Yazdanbakhsh, Ashkan Moradifirouzabadi, Zheng Li, and Mingu Kang. Sparse attention acceleration with synergistic in-memory pruning and on-chip recomputation. In 2022 55th IEEE/ACM International Symposium on Microarchi- tecture (MICRO), pages 744–762. IEEE, 2022

    2022

  67. [76]

    vision-transformers-cifar10: Training vision transformers (vit) and related models on cifar-10

    Kentaro Yoshioka. vision-transformers-cifar10: Training vision transformers (vit) and related models on cifar-10. https://github.com/kentaroy47/vision- transformers-cifar10, 2024

    2024

  68. [77]

    Rram for compute- in-memory: From inference to training.IEEE Transactions on Circuits and Systems I: Regular Papers, 68(7):2753–2765, 2021

    Shimeng Yu, Wonbo Shim, Xiaochen Peng, and Yandong Luo. Rram for compute- in-memory: From inference to training.IEEE Transactions on Circuits and Systems I: Regular Papers, 68(7):2753–2765, 2021

    2021

  69. [78]

    Neuro-inspired computing chips

    Wenqiang Zhang, Bin Gao, Jianshi Tang, Peng Yao, Shimeng Yu, Meng-Fan Chang, Hoi-Jun Yoo, He Qian, and Huaqiang Wu. Neuro-inspired computing chips. Nature electronics, 3(7):371–382, 2020

    2020

  70. [79]

    Transpim: A memory-based acceleration via software-hardware co-design for transformer

    Minxuan Zhou, Weihong Xu, Jaeyoung Kang, and Tajana Rosing. Transpim: A memory-based acceleration via software-hardware co-design for transformer. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 1071–1085. IEEE, 2022

    2022

  71. [80]

    The future of electronics based on memristive systems

    Mohammed A Zidan, John Paul Strachan, and Wei D Lu. The future of electronics based on memristive systems. Nature electronics, 1(1):22–29, 2018

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.