Pith. sign in

REVIEW 5 major objections 5 minor 37 references

CD-PIM aims to accelerate low-batch LLM inference on edge devices by partitioning each LPDDR5 bank into four pseudo-banks, claiming up to 11.42x speedup over a GPU-only baseline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:48 UTC pith:URXAN7TZ

load-bearing objection A plausible LPDDR5 PIM architecture with a genuinely new combination, but the load-bearing 4x bandwidth and the speedups that ride on it are asserted in simulation rather than demonstrated at the circuit level. the 5 major comments →

arxiv 2601.12298 v1 pith:URXAN7TZ submitted 2026-01-18 cs.AR

CD-PIM: A High-Bandwidth and Compute-Efficient LPDDR5-Based PIM for Low-Batch LLM Acceleration on Edge-Device

Ye Lin , Chao Fang , Xiaoyong Song , Qi Wu , Anying Jiang , Yichuan Bai , Li Du This is my paper
classification cs.AR
keywords processing-in-memoryLPDDR5LLM inferenceGEMVedge devicepseudo-banklow-batchmemory bandwidth
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a processing-in-memory (PIM) architecture built on LPDDR5 can break the memory-bandwidth ceiling that limits low-batch LLM inference on edge devices. Its central move is to split each physical DRAM bank into four pseudo-banks that can be activated at the same time, multiplying internal bandwidth fourfold, and to add a pipelined compute unit that runs at twice the memory clock. In simulation, this yields average speedups of 11.42x over a GPU-only baseline and 4.25x over a prior PIM design for single-batch workloads, plus a separate interleaving mode that overlaps prefill and decode for compute-heavy cases. The paper matters because decode latency is bandwidth-bound and edge devices have few banks, so the standard bank-parallelism trick does not scale.

Core claim

On its own terms, the paper's discovery is that segmenting the global bitline and its sense amplifiers into four independently addressed pseudo-banks removes the bank-count limit on internal bandwidth: what was one bank becomes four concurrently readable quarters. Combined with a compute-efficient CU that serially feeds weight data and performs inner- and outer-product operations at 400 MHz—twice the internal LPDDR5 clock—each bank can sustain a GEMV computation at significantly higher throughput. The paper reports that in its high-bandwidth mode, this architecture accelerates memory-intensive LLaMA workloads by 4.48x–18.6x over a GPU baseline depending on platform and model, and that a seco

What carries the argument

The key object is the pseudo-bank (Pbank): a quarter of a DRAM bank created by splitting the global bitline into left/right and upper/lower segments with isolation transistors, so that four Pbanks in one physical bank can be activated simultaneously. The accompanying compute-efficient CU, placed per bank and operating at twice the memory clock, reads 32 B per cycle from each side, buffers input vectors and partial sums, and can execute both outer-product (for the K-cache) and inner-product (for the V-cache) GEMV dataflows. These two mechanisms together convert bank-level parallelism into a 4x internal-bandwidth multiplier and a 2x compute-capacity multiplier over prior designs.

Load-bearing premise

The load-bearing assumption is that a single physical DRAM bank can be split into four pseudo-banks that all activate on the same command with negligible timing and area overhead; the paper asserts this through its simulator modification rather than demonstrating it with timing analysis or a fabricated test chip.

What would settle it

A timing-accurate DRAM simulation or silicon measurement showing that activating all four pseudo-banks in one physical bank introduces extra row/bank access conflicts, increases read latency beyond the modeled single-cycle activation, or pushes the area overhead above the claimed 0.8% would falsify the central bandwidth claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the 4x pseudo-bank bandwidth is realized, decode-stage GEMV latency—the dominant cost in long-output LLM inference—is cut by roughly the same factor, which is why the reported speedups on memory-intensive workloads are large.
  • The column-wise mapping for the K-cache and row-wise mapping for the V-cache let both CUs stay busy across attention heads, avoiding the one-CU bottleneck when a single new token appends a column to the cache.
  • The area/power estimate (0.8% of a 32 Gb LPDDR5 die, 144 mW total) implies the bandwidth gain comes at a cost that is small enough to keep the design viable for edge devices.
  • LBIM's overlap of GEMV and GEMM suggests that PIM-equipped memory can hide prefill latency behind decode work, reducing end-to-end latency for compute-intensive workloads with long input sequences.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The pseudo-bank splitting technique, if physically validated, could generalize to other DRAM families or to PIM designs with more banks, potentially making the bandwidth multiplier less dependent on the total bank count.
  • Because the speedup numbers rely on a modified DRAM simulator rather than silicon, a natural testable extension is a timing-accurate model or prototype that confirms the four Pbanks can be activated concurrently without extra row cycles or bank conflicts.
  • The LBIM result hints at a broader principle: co-scheduling memory-bound and compute-bound work on the same PIM memory could reduce the utilization gap that blocked-mode PIM architectures suffer, which might apply to other latency-sensitive workloads beyond LLMs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes CD-PIM, a bank-level digital PIM architecture for LPDDR5-based edge devices, targeting the memory-bandwidth bottleneck of low-batch LLM decode (GEMV). The key idea is to split each DRAM bank into four pseudo-banks by segmenting the global bitline and BLSA, allegedly yielding a 4x internal bandwidth improvement. On top of this, the paper introduces two modes: HBCEM, which activates all four pseudo-banks to accelerate GEMV, and LBIM, which overlaps PIM GEMV with processor GEMM by partitioning the pseudo-banks. The authors also propose a two-CU-per-bank pipelined datapath and a hybrid K/V-cache mapping strategy. Evaluation is performed with a modified Ramulator2 simulator on LPDDR5 configurations matching the Jetson AGX Orin and iPhone 15 Pro, reporting speedups of 11.42x over GPU-only and 4.25x over AttAcc, plus a 1.12x LBIM-vs-HBCEM speedup. The paper concludes that area/power overhead of the CUs is small.

Significance. If the architectural claims hold, CD-PIM addresses a real and timely problem: edge LLM decode is bandwidth-bound at low batch sizes, and existing bank-level PIM designs achieve only limited internal bandwidth. The two-mode design (HBCEM/LBIM) is a sensible way to handle both memory- and compute-intensive workloads, and the proposed K/V mapping strategies are clearly motivated. The paper also provides a concrete CU design with area/power numbers from synthesis, which is a strength. However, the central 4x bandwidth claim is asserted rather than physically demonstrated, and the Ramulator2 modifications are neither released nor specified in enough detail to verify that the simulated speedups are not simply a direct artifact of assuming four simultaneous activations per bank. The INT8 accuracy claim is also unsupported, which weakens the comparison to higher-precision baselines. These issues are load-bearing for the headline results, so the paper is not ready for acceptance in its current form.

major comments (5)
  1. [Sec. III-A and Sec. IV-A] The central claim that subdividing each bank into four pseudo-banks yields a 4x internal bandwidth is asserted rather than demonstrated. Splitting the global bitline and BLSA creates independent column-access paths, but concurrent row activation of four Pbanks requires independent row-decode/wordline-driver resources. The text states that upper/lower banks 'can be accessed simultaneously by the row decoder' (Sec. III-A) without stating whether a second row decoder is added or what its timing/area cost is. Since all reported speedups (e.g., 11.42x and 4.25x in the abstract) scale with this 4x assumption, please provide circuit-level evidence or a realistic timing model. In addition, the Ramulator2 modification is described only as 'modifying Ramulator2' (Sec. IV-A); if the simulator simply permits four Pbank activations per bank per cycle, the bandwidth gain is an input, not a finding. Th
  2. [Sec. III and Sec. IV-B] The claim that INT8 precision 'does not lead to any noticeable degradation in LLM inference accuracy' is unsupported. No accuracy measurements, quantization scheme, or reference are provided. The GPU-only baseline is presumably running in higher precision, so the speedups reported in Figs. 5-7 may not be apples-to-apples. Please report task-level accuracy (e.g., perplexity or downstream-task metrics) for the evaluated workloads in both precisions, or cite a prior study demonstrating that this quantization scheme preserves accuracy for the specific models used.
  3. [Sec. III-B, Table II] LBIM requires simultaneous GEMV PIM operations and processor GEMM accesses to disjoint halves of the same bank. The paper does not analyze DRAM command scheduling or row-buffer conflicts in this mode: the processor's GEMM may activate a row in two Pbanks while PIM activates different rows in the other two Pbanks. Standard LPDDR5 timing constraints (e.g., tRC, tRRD, activation limits) are not shown to be satisfied. This is load-bearing for the reported 1.12x LBIM-vs-HBCEM speedup (abstract). Please provide a detailed memory-controller scheduling analysis or a Ramulator2 model that tracks per-Pbank state and commands.
  4. [Sec. IV-A and Fig. 5] The comparison with AttAcc is under-specified. AttAcc is an HBM-based cloud PIM design [13], while the evaluation is on LPDDR5-based edge devices. It is unclear whether AttAcc's 242 TB/s internal bandwidth was scaled down to the LPDDR5 configuration or whether it is simulated with the same number of banks and PIM logic. Without this information, the reported 4.25x average speedup over AttAcc could reflect an unfair baseline. Please specify the adapted AttAcc configuration (number of banks, per-bank bandwidth, CU model) used in the Ramulator2 experiments.
  5. [Sec. IV-C and Fig. 8] The area/power overhead is computed for the CU only. The proposed Pbank partitioning requires additional isolation transistors, segmented BLSAs, and potentially additional row-decode/wordline-driver circuitry; these are excluded from Fig. 8. The conclusion that the cost is only 0.8% of a 32 Gb die area is therefore not established. Please provide a full overhead estimate for the Pbank-related peripheral changes, or clearly state the assumptions under which they are negligible.
minor comments (5)
  1. [Abstract and Sec. I] The fourth contribution is introduced with 'Forth'; this should be 'Fourth'.
  2. [Sec. III-C] The notation '1B' is ambiguous: it could mean 1 byte or 1 bit. Please use an unambiguous abbreviation (e.g., '1 B' for byte) and define it at first use.
  3. [Sec. IV-A] The sentence 'implemented on top of 4GB LPDDR5 by modifying Ramulator2' is unclear; later the paper mentions 4 and 16 LPDDR5 dies giving 16 GB and 64 GB. Please specify the per-die density and the exact Ramulator2 configuration (timing parameters, address mapping, bank count).
  4. [Sec. II-A] The parameters Lin and Lout are used in the evaluation but not defined in the text. Please define them as input and output sequence lengths, respectively.
  5. [Fig. 2] The isolation transistors and the segmentation of the global BLSA are difficult to discern in the figure. A magnified callout with explicit labels would improve readability.

Circularity Check

0 steps flagged

No significant circularity: the reported speedups are simulation outputs, not definitional consequences of fitted inputs or a self-citation chain.

full rationale

CD-PIM's central claims are performance numbers obtained from a Ramulator2-based simulation of a proposed LPDDR5 PIM, not from a mathematical derivation that reduces to its own inputs. The paper does not fit any parameter to the reported speedups; HBCEM, LBIM, the two-CU compute pipeline, and the K/V mapping are stated as architecture/mapping choices, and the speedups are measured relative to GPU and AttAcc baselines. The only self-citation ([26], for the standard fact that transformers contain cascaded decoder layers) is background and not load-bearing. The most plausible non-circularity concern is that the claimed 4× bandwidth from partitioning each bank into four Pbanks is an asserted architectural property that the modified simulator presumably embodies; if so it is an unvalidated modeling assumption rather than a measured result, but it is an input design assumption, not a fitted value or an equation-level equivalence, so it does not constitute a circular step under the required standard.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The central speedup claims rest mainly on hand-chosen design parameters (four pseudo-banks, 400 MHz CUs, 32B datapath, INT8) and on assumptions that LPDDR5 can be physically split into four concurrently accessible banks and that Ramulator2 reproduces that behavior. No fitted parameters are present; there is also no released artifact or silicon.

free parameters (4)
  • Pseudo-bank count per bank = 4
    Hand-chosen to claim 4x bank-level bandwidth; speedup results depend on it.
  • CU frequency = 400 MHz
    Set to twice LPDDR5 internal clock; throughput and area numbers depend on it.
  • CU datapath width per cycle = 32 B
    Assumed for pipelined MAC; determines per-bank GEMV throughput.
  • INT8 precision = 8-bit
    All input/weight data precision; accuracy claim is not backed by experiment.
axioms (5)
  • ad hoc to paper A DRAM bank can be divided into four independently addressable/activatable pseudo-banks by splitting GBL and BLSA and inserting isolation transistors, without breaking LPDDR5 timing or incurring significant overhead.
    Fig. 2 and Sec. III-A assume this; no validation.
  • domain assumption Ramulator2, modified to model these Pbanks, faithfully reflects real LPDDR5 bandwidth and latency.
    Sec. IV-A uses Ramulator2; no calibration or traces.
  • domain assumption INT8 weights/activations do not noticeably degrade LLM accuracy.
    Sec. III states this with no experiment or citation.
  • ad hoc to paper In LBIM, the processor's GEMM and PIM GEMV can access disjoint halves of the bank concurrently with no additional conflict or area overhead.
    Table II instructions assume this; no cycle-level validation.
  • domain assumption Area/power of the new CU measured at TSMC 28nm can be added to a 32Gb LPDDR5 die with 0.8% area and 144 mW overhead.
    Sec. IV-C extrapolates from synthesis; no full-chip integration.
invented entities (1)
  • Four pseudo-banks per bank via segmented global bitlines no independent evidence
    purpose: Provide 4x internal bandwidth for GEMV
    Only simulated in Ramulator2; no fabricated part or timing model.

pith-pipeline@v1.3.0-alltime-deepseek · 9940 in / 11777 out tokens · 118029 ms · 2026-08-03T09:48:22.840494+00:00 · methodology

0 comments
read the original abstract

Edge deployment of low-batch large language models (LLMs) faces critical memory bandwidth bottlenecks when executing memory-intensive general matrix-vector multiplications (GEMV) operations. While digital processing-in-memory (PIM) architectures promise to accelerate GEMV operations, existing PIM-equipped edge devices still suffer from three key limitations: limited bandwidth improvement, component under-utilization in mixed workloads, and low compute capacity of computing units (CUs). In this paper, we propose CD-PIM to address these challenges through three key innovations. First, we introduce a high-bandwidth compute-efficient mode (HBCEM) that enhances bandwidth by dividing each bank into four pseudo-banks through segmented global bitlines. Second, we propose a low-batch interleaving mode (LBIM) to improve component utilization by overlapping GEMV operations with GEMM operations. Third, we design a compute-efficient CU that performs enhanced GEMV operations in a pipelined manner by serially feeding weight data into the computing core. Forth, we adopt a column-wise mapping for the key-cache matrix and row-wise mapping for the value-cache matrix, which fully utilizes CU resources. Our evaluation shows that compared to a GPU-only baseline and state-of-the-art PIM designs, our CD-PIM achieves 11.42x and 4.25x speedup on average within a single batch in HBCEM mode, respectively. Moreover, for low-batch sizes, the CD-PIM achieves an average speedup of 1.12x in LBIM compared to HBCEM.

Figures

Figures reproduced from arXiv: 2601.12298 by Anying Jiang, Chao Fang, Li Du, Qi Wu, Xiaoyong Song, Ye Lin, Yichuan Bai.

Figure 1
Figure 1. Figure 1: (a) Conventional PIM architecture with a single CU; (b) CD-PIM [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Details of the CD-PIM architecture, where each bank is divided into [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Detailed data mapping of the CD-PIM: (a) K-cache matrix data mapping [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Timing diagrams of LLM inference: (a) GPU with memory-intensive [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Normalized performance of LLaMA-1B, -7B, and -13B (batch size [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: The area and power breakdown of CU. 1.01×–1.46× and 1.01×–1.44× over HBCEM, respectively, when Lout ranges from 2 to 128, because the decode stage latency increases to nearly half of the total inference latency, as shown in [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗
Figure 7
Figure 7. Figure 7: Normalized performance of the Apple iPhone 15 Pro equipped with [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith

  1. [1]

    Specpim: Accelerating speculative inference on pim- enabled system via architecture-dataflow co-exploration,

    C. Liet al., “Specpim: Accelerating speculative inference on pim- enabled system via architecture-dataflow co-exploration,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), V olume 3, 2024, p. 950–965

    2024

  2. [2]

    Opt: Open pre-trained transformer language models,

    S. Zhanget al., “Opt: Open pre-trained transformer language models,”

  3. [3]

    Codegen: An open large language model for code with multi-turn program synthesis,

    E. Nijkampet al., “Codegen: An open large language model for code with multi-turn program synthesis,” 2023. [Online]. Available: https://arxiv.org/abs/2203.13474

    Pith/arXiv arXiv 2023

  4. [4]

    Make llm inference affordable to everyone: Augmenting gpu memory with ndp-dimm,

    L. Liuet al., “Make llm inference affordable to everyone: Augmenting gpu memory with ndp-dimm,” inIEEE International Symposium on High Performance Computer Architecture (HPCA), 2025, pp. 1751–1765

    2025

  5. [5]

    Fold-pim: A cost-efficient lpddr5-based pim for on-device slms,

    K. Jeunet al., “Fold-pim: A cost-efficient lpddr5-based pim for on-device slms,”IEEE Computer Architecture Letters (CAL), vol. 24, no. 1, pp. 185–188, 2025

    2025

  6. [6]

    Dimm-link: Enabling efficient inter-dimm communication for near-memory processing,

    Z. Zhouet al., “Dimm-link: Enabling efficient inter-dimm communication for near-memory processing,” inIEEE International Symposium on High- Performance Computer Architecture (HPCA), 2023, pp. 302–316

    2023

  7. [7]

    Are llms ready for practical adoption for assertion generation?

    V . Pulavarthiet al., “Are llms ready for practical adoption for assertion generation?” inDesign, Automation & Test in Europe Conference (DATE), 2025, pp. 1–7

    2025

  8. [8]

    Nvcim-pt: An nvcim-assisted prompt tuning framework for edge llms,

    R. Qinet al., “Nvcim-pt: An nvcim-assisted prompt tuning framework for edge llms,” inDesign, Automation & Test in Europe Conference (DATE), 2025, pp. 1–7

    2025

  9. [9]

    Nora: Noise-optimized rescaling of llms on analog compute-in-memory accelerators,

    Y . Houet al., “Nora: Noise-optimized rescaling of llms on analog compute-in-memory accelerators,” in2025 Design, Automation & Test in Europe Conference (DATE), 2025, pp. 1–7

    2025

  10. [10]

    Aster: Adaptive dynamic layer-skipping for efficient transformer inference via markov decision process,

    F. Liuet al., “Aster: Adaptive dynamic layer-skipping for efficient transformer inference via markov decision process,” inProceedings of the 33rd ACM International Conference on Multimedia (MM), 2025, pp. 11 853–11 861

    2025

  11. [11]

    Dracc: a dram based accelerator for accurate cnn inference,

    Q. Denget al., “Dracc: a dram based accelerator for accurate cnn inference,” in55th ACM/ESDA/IEEE Design Automation Conference (DAC), 2018, pp. 1–6

    2018

  12. [12]

    Mcdram v2: In-dynamic random access memory systolic array accelerator to address the large model problem in deep neural networks on the edge,

    S. Choet al., “Mcdram v2: In-dynamic random access memory systolic array accelerator to address the large model problem in deep neural networks on the edge,”IEEE Access, vol. 8, pp. 135 223–135 243, 2020

    2020

  13. [13]

    Attacc! unleashing the power of pim for batched transformer-based generative model inference,

    J. Parket al., “Attacc! unleashing the power of pim for batched transformer-based generative model inference,” inProceedings of the 29th ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems (ASPLOS), V olume 2, 2024, p. 103–119

    2024

  14. [14]

    An lpddr-based cxl-pnm platform for tco-efficient infer- ence of transformer-based large language models,

    S.-S. Parket al., “An lpddr-based cxl-pnm platform for tco-efficient infer- ence of transformer-based large language models,” inIEEE International Symposium on High-Performance Computer Architecture (HPCA), 2024, pp. 970–982

    2024

  15. [15]

    Pipepim: Maximizing computing unit utilization in ml- oriented digital pim by pipelining and dual buffering,

    T. Jeonget al., “Pipepim: Maximizing computing unit utilization in ml- oriented digital pim by pipelining and dual buffering,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol. 43, no. 12, pp. 4585–4598, 2024

    2024

  16. [16]

    A 1ynm 1.25v 8gb, 16gb/s/pin gddr6-based accelerator-in- memory supporting 1tflops mac operation and various activation functions for deep-learning applications,

    S. Leeet al., “A 1ynm 1.25v 8gb, 16gb/s/pin gddr6-based accelerator-in- memory supporting 1tflops mac operation and various activation functions for deep-learning applications,” inIEEE International Solid-State Circuits Conference (ISSCC), vol. 65, 2022, pp. 1–3

    2022

  17. [17]

    Pimoe: Towards efficient moe transformer deployment on npu-pim system through throttle-aware task offloading,

    L. Wuet al., “Pimoe: Towards efficient moe transformer deployment on npu-pim system through throttle-aware task offloading,” in2025 62nd ACM/IEEE Design Automation Conference (DAC). IEEE, 2025, pp. 1–7

    2025

  18. [18]

    Plain: Leveraging high internal bandwidth in pim for accel- erating large language model inference via mixed-precision quantization,

    Y . Huet al., “Plain: Leveraging high internal bandwidth in pim for accel- erating large language model inference via mixed-precision quantization,” in2025 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 2025, pp. 1–9

    2025

  19. [19]

    P3-llm: An integrated npu-pim accelerator for llm infer- ence using hybrid numerical formats,

    Y . Chenet al., “P3-llm: An integrated npu-pim accelerator for llm infer- ence using hybrid numerical formats,”arXiv preprint arXiv:2511.06838, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Um-pim: Dram-based pim with uniform & shared mem- ory space,

    Y . Zhaoet al., “Um-pim: Dram-based pim with uniform & shared mem- ory space,” in2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 2024, pp. 644–659

    2024

  21. [21]

    Paise: Pim-accelerated inference scheduling engine for transformer-based llm,

    H. Leeet al., “Paise: Pim-accelerated inference scheduling engine for transformer-based llm,” inIEEE International Symposium on High Per- formance Computer Architecture (HPCA), 2025, pp. 1707–1719

    2025

  22. [22]

    Memory-centric computing with sk hynix’s domain- specific memory,

    Y . Kwonet al., “Memory-centric computing with sk hynix’s domain- specific memory,” inIEEE Hot Chips 35 Symposium (HCS), 2023, pp. 1–26

    2023

  23. [23]

    Ianus: Integrated accelerator based on npu-pim uni- fied memory system,

    M. Seoet al., “Ianus: Integrated accelerator based on npu-pim uni- fied memory system,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), V olume 3, 2024, p. 545–560

    2024

  24. [24]

    Nvidia jetson agx orin series data sheet,

    NVIDIA, “Nvidia jetson agx orin series data sheet,” https: //www.diamondsystems.com/files/binaries/Jetson%20AGX%20Orin% 20DS10662-001%20v1.2.pdf, 2022, accessed: 2025-09-14

    2022

  25. [25]

    Apple announces m3 soc family: M3, m3 pro, and m3 max make their marks,

    R. Smith, “Apple announces m3 soc family: M3, m3 pro, and m3 max make their marks,” https://www.anandtech.com/show/21116/ appleannounces-m3-soc-family-m3-m3-pro-and-m3-max-make-their-marks, 2023, accessed: 2025-09-14

    2023

  26. [26]

    Anda: Unlocking efficient llm inference with a variable- length grouped activation data format,

    C. Fanget al., “Anda: Unlocking efficient llm inference with a variable- length grouped activation data format,” inIEEE International Symposium on High Performance Computer Architecture (HPCA), 2025, pp. 1467– 1481

    2025

  27. [27]

    Apt-llm: Exploiting arbitrary-precision tensor core comput- ing for llm acceleration,

    S. Maet al., “Apt-llm: Exploiting arbitrary-precision tensor core comput- ing for llm acceleration,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 2025

    2025

  28. [28]

    AIOS: LLM agent operating system,

    K. Meiet al., “AIOS: LLM agent operating system,” inSecond Conference on Language Modeling (COLM), 2025. [Online]. Available: https://openreview.net/forum?id=L4HHkCDz2x

    2025

  29. [29]

    Pim-enabled instructions: A low-overhead, locality- aware processing-in-memory architecture,

    J. Ahnet al., “Pim-enabled instructions: A low-overhead, locality- aware processing-in-memory architecture,” inACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), 2015, pp. 336–348

    2015

  30. [30]

    Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,

    G. Heoet al., “Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), V olume 3, 2024, p. 722–737

    2024

  31. [31]

    H2-llm: Hardware-dataflow co-exploration for heteroge- neous hybrid-bonding-based low-batch llm inference,

    C. Liet al., “H2-llm: Hardware-dataflow co-exploration for heteroge- neous hybrid-bonding-based low-batch llm inference,” inProceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA), 2025, p. 194–210

    2025

  32. [32]

    Facil: Flexible dram address mapping for soc-pim cooperative on-device llm inference,

    S. H. Seoet al., “Facil: Flexible dram address mapping for soc-pim cooperative on-device llm inference,” inIEEE International Symposium on High Performance Computer Architecture (HPCA), 2025, pp. 1720– 1733

    2025

  33. [33]

    A 16-gb 37-gb/s gddr7 dram with pam3-optimized trx equalization and zq calibration,

    S.-Y . Choet al., “A 16-gb 37-gb/s gddr7 dram with pam3-optimized trx equalization and zq calibration,”IEEE Journal of Solid-State Circuits (JSSC), vol. 60, no. 1, pp. 184–196, 2025

    2025

  34. [34]

    Dh-pim: Maximizing computing unit utilization in dig- ital pim by dual half mode extension,

    B. J. Kimet al., “Dh-pim: Maximizing computing unit utilization in dig- ital pim by dual half mode extension,”IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems (TCAD), 2025

    2025

  35. [35]

    Ramulator 2.0: A modern, modular, and extensible dram simulator,

    H. Luoet al., “Ramulator 2.0: A modern, modular, and extensible dram simulator,”IEEE Computer Architecture Letters (RAL), vol. 23, no. 1, pp. 112–116, 2024

    2024

  36. [36]

    Llama: Open and efficient foundation language models,

    H. Touvronet al., “Llama: Open and efficient foundation language models,” 2023. [Online]. Available: https://arxiv.org/abs/2302.13971

    Pith/arXiv arXiv 2023

  37. [2022]

    Available: https://arxiv.org/abs/2205.01068

    [Online]. Available: https://arxiv.org/abs/2205.01068

    Pith/arXiv arXiv