ABI: A tightly integrated, unified, sparsity-aware, reconfigurable, compute near-register file/cache GPU architecture with light-weight softmax for deep learning, linear algebra, and Ising compute

Jaydeep P. Kulkarni; Siddhartha Raman Sundara Raman

arxiv: 2602.14262 · v3 · pith:S2K2OBB6new · submitted 2026-02-15 · 💻 cs.AR

ABI: A tightly integrated, unified, sparsity-aware, reconfigurable, compute near-register file/cache GPU architecture with light-weight softmax for deep learning, linear algebra, and Ising compute

Siddhartha Raman Sundara Raman , Jaydeep P. Kulkarni This is my paper

Pith reviewed 2026-05-15 21:52 UTC · model grok-4.3

classification 💻 cs.AR

keywords near-memory computingGPU architecturesparsity-aware designsoftmax accelerationenergy-efficient computingreconfigurable hardwaredeep learning accelerationIsing computing

0 comments

The pith

A tightly integrated near-memory GPU architecture called ABI achieves 6-16 times speedup and 6-13 times energy savings on convolutional neural networks, graph networks, linear programming, large language models, and Ising workloads compared

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces ABI, a unified GPU architecture that moves custom compute close to the register file and cache to exploit data sparsity efficiently. It incorporates a specialized sparsity-aware circuit and a lightweight softmax implementation to reduce energy use. The design supports reconfigurable integer precision up to 16 bits and demonstrates strong scaling across different workload sizes. A sympathetic reader would care because this promises large performance and efficiency gains for a wide range of modern computing tasks without requiring entirely new hardware paradigms.

Core claim

The paper claims that by tightly integrating sparsity-aware compute near the register file and cache along with a lightweight softmax circuit, a reconfigurable GPU architecture can deliver 6 to 16 times speedup and 6 to 13 times energy savings across diverse workloads including CNNs, GCNs, linear programming, LLMs, and Ising models, while also achieving 4.5 times speedup on next-generation systems like MI300 and Blackwell.

What carries the argument

The ABI architecture, a tightly integrated unified near-memory design with sparsity-aware circuits and lightweight softmax placed near the register file and cache to enable reconfigurable compute up to INT16.

If this is right

ABI provides about 1.5 times energy savings from the sparsity-aware near-memory circuit.
The lightweight softmax circuit contributes about 1.6 times energy savings.
The architecture supports dynamic resolution updates and scales efficiently across problem sizes.
ABI-enabled MI300 and Blackwell systems achieve about 4.5 times speedup over baseline versions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the overheads remain low, similar near-register-file compute could be applied to other processor types like CPUs or accelerators for matrix operations.
The reconfigurability up to INT16 suggests potential for mixed-precision computing that adapts to different parts of a neural network dynamically.
Extending this to even sparser or quantized models could yield further gains in edge computing scenarios.

Load-bearing premise

The design assumes the custom sparsity-aware near-memory circuit and lightweight softmax can be added with negligible area, latency, and power overheads while keeping the architecture scalable and reconfigurable.

What would settle it

Fabricating a prototype chip and measuring its actual area overhead, power consumption, and performance on the claimed workloads would confirm or refute the negligible overhead assumption if the measured values deviate significantly from the modeled savings.

Figures

Figures reproduced from arXiv: 2602.14262 by Jaydeep P. Kulkarni, Siddhartha Raman Sundara Raman.

**Figure 1.** Figure 1: Limitations of existing accelerators (red/first column), proposed design changes to realize ABI (green/second column), resultant energy savings, area, efficiency using ABI (green/third column) R2) Problem resolutions vary within each application (e.g., 1-16 bits in Ising/CNN), so the optimal Bit-Serial/Bit-Parallel (BS/BP) and Element-Serial/Element-Parallel (ES/EP) mode depends on the problem, not the app… view at source ↗

**Figure 2.** Figure 2: ABI enabled a) tightly integrated GPU including dispatcher, compute unit, L2 cache, b) compute unit c) wavefront fetch, pool d) decode, issue e) register file f) load/store units. g) Near-memory(NM) / Near-RF(NRF) logic floorplan h) Programmable registers i) Legend a custom sparsity-aware circuit to achieve ∼1.8x energy savings. This article presents the first tightly integrated, sparsity-aware, reconfigur… view at source ↗

**Figure 5.** Figure 5: a) Die photograph b) Measurement setup c) Area, power breakdown for CNN, LP, GCN, Ising, LLM across RCE, sparsity, TH, CA, S, PR d) Offline program flow e) Programming model f) Benchmarks g) Parameters of ABI B. Unified architecture Oscilloscope captures (Fig.6b–e) validate NRF functionality across different applications, using identical inputs, while mapping onto ABI differently (Fig.6a) but producing wor… view at source ↗

**Figure 6.** Figure 6: a) Hardware mapping for unified architecture, b) Oscilloscope capture with output values circled for b) CNN c) Ising d) LP e) GCN/LLM. ABI, (ABI+BASE) Speedup, Energy efficiency, energy savings from sparsity awareness for f) CNN g) Ising h) GCN i) LP j) LLM wrt BASE (4-neighbors): each bank’s output is summed, scaled and yields 2. LLM: Key and Value matrices reside in memory, and the Query matrix is stored… view at source ↗

**Figure 8.** Figure 8: Examples of a) CNN convolution, b) Linear programming c) Transfomer engine, attention, add and norm d) GCN combination, aggregation e) Ising compute VII. CONCLUSION We present the first unified, sparsity-aware design that integrates reconfigurable near-memory compute into a GPU for CNNs, Ising compute, LPs, transformers, and GCN in TSMC65nm. We achieve speedups of 6-16x and energy savings of 6-13x over MIA… view at source ↗

read the original abstract

We present a tightly integrated and unified near-memory GPU architecture that delivers 6 to 16 times speedup and 6 to 13 times energy savings across Convolutional Neural Networks, Graph Convolutional Networks, Linear Programming, Large Language Models, and Ising workloads compared to MIAOW GPU. The design includes a custom sparsity-aware near-memory circuit providing about 1.5 times energy savings, and a lightweight softmax circuit providing about 1.6 times energy savings. The architecture supports reconfigurable compute up to INT16 with dynamic resolution updates and scales efficiently across problem sizes. ABI-enabled MI300 and Blackwell systems achieve about 4.5 times speedup over baseline MI300 and Blackwell.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

ABI combines near-register-file compute with sparsity awareness and a lightweight softmax in one reconfigurable unit, but the 6-16x claims rest on unshown integration overheads.

read the letter

The main point is a GPU architecture that moves simple compute operations right next to the register file and cache, adds sparsity detection in the same path, and includes a stripped-down softmax unit, all reconfigurable up to INT16 with dynamic updates. The authors report 6-16x speedup and 6-13x energy reduction versus MIAOW across CNNs, GCNs, linear programming, LLMs, and Ising models, plus 4.5x on MI300 and Blackwell baselines. The custom sparsity circuit is credited with roughly 1.5x energy savings and the softmax with 1.6x, which they combine into the aggregate figures. The unified reconfigurability across those workloads is the clearest new element; most prior near-memory GPU work keeps these pieces separate. The paper does a reasonable job framing the design around real data-movement bottlenecks in mixed linear-algebra and ML loads. The soft spot is the missing breakdown of area, latency, and power after the new circuits are integrated. The abstract and claims give only the final speedups with no post-placement numbers, no sensitivity to problem size, and no comparison of overhead versus the claimed gains. If the added logic exceeds a few percent in any dimension, the net 6-16x numbers will not survive. The comparison baseline is MIAOW rather than a current production GPU in every case, which also limits how far the gains can be extrapolated. This is for architects working on near-memory extensions for DL and scientific computing. A reader who needs concrete ideas for sparsity-aware register-file logic or lightweight activation units could extract useful blocks even if the overall numbers require more verification. It deserves peer review so the implementation details and overhead measurements can be examined directly.

Referee Report

2 major / 0 minor

Summary. The manuscript presents ABI, a tightly integrated, unified, sparsity-aware, reconfigurable GPU architecture with compute near the register file/cache and a lightweight softmax unit. It claims 6-16x speedup and 6-13x energy savings versus the MIAOW GPU across CNNs, GCNs, linear programming, LLMs, and Ising workloads, plus ~4.5x speedup on MI300 and Blackwell systems. The design includes a custom near-memory circuit (~1.5x energy savings) and softmax circuit (~1.6x energy savings), supports dynamic INT16 resolution, and scales across problem sizes.

Significance. If the performance and energy claims hold under detailed evaluation, the work could meaningfully advance domain-specific GPU architectures by unifying sparse near-memory compute with reconfigurability for mixed workloads. The emphasis on negligible integration overheads and cross-domain applicability addresses real challenges in modern accelerators. However, the absence of any supporting data, simulations, or breakdowns in the manuscript prevents assessment of whether these gains are realizable.

major comments (2)

[Abstract] Abstract: The central performance claims (6-16x speedup, 6-13x energy savings vs. MIAOW; ~4.5x on MI300/Blackwell) are asserted without any simulation results, area/power/latency breakdowns, error analysis, or workload-specific data. This absence makes the claims impossible to evaluate and directly undermines the soundness of the primary contribution.
[Abstract] Abstract: The design premise that the sparsity-aware near-memory circuit and lightweight softmax integrate with negligible area, latency, and power overheads while preserving INT16 reconfigurability and scalability is stated without any quantitative post-placement-and-routing metrics or sensitivity analysis. If these overheads are non-negligible, the net speedup and energy figures cannot hold.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thorough review and constructive comments on the ABI manuscript. We agree that the current version requires additional supporting evidence to allow proper evaluation of the performance and energy claims, as well as quantitative metrics for integration overheads. We will revise the manuscript to incorporate the requested simulation results, breakdowns, and analyses.

read point-by-point responses

Referee: [Abstract] Abstract: The central performance claims (6-16x speedup, 6-13x energy savings vs. MIAOW; ~4.5x on MI300/Blackwell) are asserted without any simulation results, area/power/latency breakdowns, error analysis, or workload-specific data. This absence makes the claims impossible to evaluate and directly undermines the soundness of the primary contribution.

Authors: We agree with this assessment. The current manuscript presents the claims without accompanying data. In the revised version, we will add comprehensive simulation results from our evaluation framework, area/power/latency breakdowns for all key components, error analysis, and workload-specific data for CNNs, GCNs, linear programming, LLMs, and Ising workloads. These additions will substantiate the 6-16x speedup and 6-13x energy savings versus MIAOW as well as the ~4.5x speedup on MI300 and Blackwell systems. revision: yes
Referee: [Abstract] Abstract: The design premise that the sparsity-aware near-memory circuit and lightweight softmax integrate with negligible area, latency, and power overheads while preserving INT16 reconfigurability and scalability is stated without any quantitative post-placement-and-routing metrics or sensitivity analysis. If these overheads are non-negligible, the net speedup and energy figures cannot hold.

Authors: We concur that quantitative evidence is essential. The revised manuscript will include post-placement-and-routing metrics from our synthesis flow, detailing the area, latency, and power overheads of the sparsity-aware near-memory circuit (providing ~1.5x energy savings) and the lightweight softmax circuit (providing ~1.6x energy savings). We will also add sensitivity analysis across problem sizes and configurations to confirm that the overheads remain negligible while preserving dynamic INT16 resolution and scalability. revision: yes

Circularity Check

0 steps flagged

No circularity: architecture claims rest on external benchmarks, not self-referential equations

full rationale

The manuscript presents a hardware architecture description and aggregate speedup/energy claims versus MIAOW, MI300, and Blackwell baselines. No equations, fitted parameters, derivations, or self-citation chains appear in the abstract or full-text placeholder. Performance numbers are presented as simulation or measurement outcomes rather than results that reduce to the paper's own inputs by construction. The design assumptions (negligible overheads for sparsity-aware circuits and softmax) are stated explicitly but are not derived from prior results within the paper itself.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only input provides no identifiable free parameters, axioms, or invented entities beyond the high-level architecture name and circuit descriptions.

pith-pipeline@v0.9.0 · 5430 in / 1276 out tokens · 37056 ms · 2026-05-15T21:52:26.277525+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

ABI’s reconfigurable compute engine (RCE) with 5-stage unified architecture... programmable registers... BIT_WID up to INT16... lightweight near-memory softmax (LWSM) circuit... approximate compute block... find-first search
IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean absolute_floor_iff_bare_distinguishability unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

sparsity-aware near-memory circuit... programmable sparsity monitor... 512 consecutive cycles... transmission-gate multiplexing

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

A complete discussion on fully reconfigurable, digital, scalable, graph and sparsity-aware near-memory accelerator for graph neural networks
cs.AR 2026-05 unverdicted novelty 5.0

NEM-GNN is a scalable DAC/ADC-less processing-in-memory architecture for GNNs that uses early compute termination, reconfigurable SoC pre-computation, and compute-as-soon-as-ready broadcast execution to deliver large ...
A comprehensive study on ILP acceleration accounting for sparsity, area, energy, data movement using near-memory architecture
cs.AR 2026-05 unverdicted novelty 5.0

SPARK is a sparsity-aware near-cache ILP accelerator that reuses L1 cache structures to deliver up to 15x speedup and 152x energy reduction versus CPUs on sparse MIPLIB workloads with 1.4% area overhead.
A comparative study on power delivery aspects of compute-in/near-memory approaches using DRAM
cs.AR 2026-04 unverdicted novelty 5.0

The survey proposes a taxonomy for PIM-induced current behaviors in DRAM and analyzes how representative techniques create voltage droop and thermal issues, along with mitigation strategies using existing DRAM mechanisms.
Emerging memory technologies at room/cryogenic temperature
cs.AR 2026-05 unverdicted novelty 1.0

Overview chapter surveying volatile and non-volatile memories including SRAM, DRAM, RRAM, MRAM, FeFET and cryogenic JJFET devices, with focus on principles, tradeoffs, and challenges.