Pith. sign in

REVIEW 4 major objections 6 minor 58 references

ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ThAME claims a 3D FeFET-NAND chiplet paired with a hierarchical NoC removes the memory-bandwidth and non-deterministic-traffic bottlenecks of Mixture-of-Experts LLM decoding, reporting up to 15.7x lower time-between-token latency and 9.8x b

desk verdict A coherent MoE accelerator design whose headline speedups rest on an internally inconsistent FeFET-NAND capacity claim. read the letter →

arxiv 2607.17074 v2 pith:UYC5UHOX submitted 2026-07-19 cs.AR cs.AIcs.DC

Pratyush Dhingra , Pramit Kumar Pal , Janardhan Rao Doppa , Partha Pratim Pande This is my paper · ORCID
classification cs.ARcs.AIcs.DC
keywords MixtureofExpertsLLMinference3DintegrationFeFET-NANDprocessing-near-memoryNetwork-on-Chipheterogeneouschiplettaillatency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ThAME is a multi-chiplet architecture for serving Mixture-of-Experts LLMs during the autoregressive decode phase. The paper argues that two distinct hardware bottlenecks—fetching massive non-contiguous expert weights and routing input-dependent scatter-gather token traffic—can be broken simultaneously by co-designing a heterogeneous 3D memory substrate and a communication fabric. It maps expert layers to high-density 3D FeFET-NAND chiplets with near-memory systolic arrays, attention to DRAM chiplets, and prefill to a compute-centric host, so that only small activation vectors cross the interposer. To handle the combinatorial space of per-batch expert-to-core assignments, it optimizes a hierarchical tree NoC with multi-objective search across all valid traffic partitions. The paper reports up to 15.7x lower time-between-token latency and 9.8x better energy efficiency versus state-of-the-art heterogeneous accelerators, with the gains growing at larger batch sizes.

What carries the argument

The load-bearing mechanisms are (1) a 3D FeFET-NAND memory chiplet whose CMOS base die holds systolic-array cores and whose hybrid-bonded vertical interface supplies ~5 TB/s, allowing expert weights to stay on-chip; (2) a hierarchical NoC—local crossbars under a two-level Butterfly Fat Tree with memory injection at the lowest level—chosen by multi-objective optimization over the valid partition space of 32 cores so that link-utilization mean and variance are minimized; and (3) an integer min-max core scheduler that assigns cores to active experts via binary search over the latency interval to equalize expert finishing times.

What would settle it

Fabricate or simulate the exact 3D FeFET-NAND chiplet with the paper's array organization and measure per-chiplet capacity, read latency, and vertical bandwidth; if capacity falls below 32 GB per chiplet (64 GB total) or read latency/energy exceed assumptions, the on-chip expert storage claim and therefore the reported speedups and energy improvements break.

Watch

Extended reading notes

Core claim

The central discovery is that the two dominant bottlenecks of MoE decode—weight-memory bandwidth and non-deterministic token-routing communication—are separable and independently addressable: heavy static expert weights belong in dense non-volatile memory with compute underneath, while dynamic token traffic belongs on a fabric designed for one-to-many scatter and many-to-one gather. ThAME keeps all expert weights on-chip (64 GB total, 32 GB per FeFET chiplet), uses a min-max scheduler to allocate systolic-array cores so active experts finish together, and uses a Butterfly Fat Tree NoC, selected from a Pareto-optimal set, to confine partial-sum traffic to local crossbars and confine scatter-g

Load-bearing premise

The entire advantage rests on the assumption that a 3D FeFET-NAND array with the stated 8 channel/4 bank/4 plane/64 block organization can actually provide 32 GB per chiplet at 10 ns read and 5 TB/s vertical bandwidth inside the 121 mm^2, 50 W base-die budget; if the capacity is lower, expert weights do not all fit on-chip and the 15.7x/9.8x results collapse.

Editorial extensions

If this is right

  • If ThAME's claims hold, MoE serving systems can remove off-chip expert-weight traffic almost entirely: only O(B x d_model) activation vectors cross the interposer per decode step instead of O(B x d_model x d_ff) weight bytes.
  • The compute-communication mismatch between expert computation and NoC latency stays below 2% for over 60% of valid spatial traffic partitions and at most 27% worst-case, meaning the scheduler's load balancing is not undone by network contention.
  • Because FeFET read latency can degrade to 50 ns with negligible effect and to 100 ns with only 2.2x expert-latency increase, the architecture is robust to device-level variability in the NVM.
  • End-to-end response time for output-dominated generation improves more with longer outputs: the TBT advantage compounds, growing from 9.8x at 512 output tokens to 10.2x at 4096 tokens in the paper's evaluation.
  • The reported 147.5 tokens/s/W and 184.3 tokens/s/kW/mm^2 performance density imply the design is competitive on energy and area, not just latency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 64 GB on-chip FeFET capacity is the key enabler, the architecture's appeal generalizes to larger MoE models only if per-chiplet capacity scales with more memory tiers; a 16-GB-per-chiplet FeFET-NAND would force some experts off-chip and likely erode the reported advantage.
  • The uniform-prior optimization over traffic partitions is a worst-case robustness choice; a deployment that knows its serving distribution could potentially do even better with a distribution-aware fabric, or worse if the real distribution concentrates on scenarios the Pareto choice did not weight.
  • The hierarchical NoC idea is not MoE-specific: any workload with unpredictable one-to-many/many-to-one communication—sparse MoE in vision or recommendation models, or distributed MoE training gradients—could reuse the same multi-objective topology search.
  • The sensitivity analysis suggests that a denser but slower NVM could still work; this opens a testable design point: trade FeFET read speed for even higher density or lower cost, and the architecture would keep most of its benefit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ThAME, a 2.5D multi-chiplet accelerator for MoE LLM inference. It combines 3D FeFET-NAND PNM chiplets for expert-weight storage and computation, 3D DRAM-PNM chiplets for attention, and a TPU-like host chiplet for prefill. A multi-objective optimization (AMOSA) over a pruned space of valid tensor/expert-parallel mappings is used to design a hierarchical crossbar/tree NoC. The authors report up to 15.7x TBT speedup and 9.8x energy-efficiency improvement over Stratum and H3D-T, plus end-to-end gains over A100 and TPU v6e, all based on cycle-accurate simulations.

Significance. The paper addresses a timely problem: MoE inference is bottlenecked not only by memory bandwidth but also by the input-dependent scatter-gather traffic that standard NoCs handle poorly. If the architectural assumptions were sound, ThAME would be a meaningful step beyond DRAM-only PNM accelerators. The paper has genuine strengths: a well-formulated MOO framework for NoC design, an external validation workload (MMLU), and ablations that separate the contributions of the memory substrate and the NoC. However, the quantitative claims currently rest on a FeFET-NAND capacity estimate that is inconsistent with the paper's own array parameters, an in-sample NoC evaluation, and a bandwidth discrepancy that undermines the sensitivity analysis. These are load-bearing issues that must be resolved before the headline speedup and energy figures can be accepted.

major comments (4)
  1. [Section V-F, Table II, Section IV-B] Section V-F states that ThAME has 64 GB aggregate on-chip FeFET-NAND (32 GB per chiplet). The array in Table II (8 channels, 4 banks, 4 planes, 64 blocks, 128 WL, 9216 BL, SLC) yields 8×4×4×64×128×9216 bits ≈ 1.21 GB per chiplet — about 26x smaller than claimed. This capacity is load-bearing: it is the stated reason ThAME can keep expert weights on-chip and avoid H3D-T's off-chip traffic, and it determines which models in Table I fit. Please provide a concrete array organization (planes/blocks, bits per cell, vertical layer count, total dies) that realizes 32 GB/chiplet and verify it under the 121 mm^2 and thermal constraints, or correct the capacity and re-run the comparisons. At minimum, report the on-chip expert footprint required by each evaluated model.
  2. [Section IV-C, Section V-C, Fig. 8] The NoC is selected by minimizing μ(d) and σ(d) over S_valid and then by choosing Solution #3 from the Pareto front using cycle-accurate latency also measured over S_valid. The 4.1x latency advantage over mesh/torus (Fig. 8) is therefore an in-sample comparison: the design has been fitted to the same traffic space on which it is evaluated. The MMLU experiment (Fig. 10) is a good external check, but it is limited to one model/workload. Please add a held-out evaluation, e.g., optimize on one random split of S_valid and evaluate on the other, or select the final design using MMLU traces and reserve S_valid for stress testing only. Report both in-sample and out-of-sample numbers.
  3. [Section V-B, Section V-E, Table IV] Section V-B says the FeFET-NAND substrate provides about 5 TB/s vertical bandwidth (Table IV). Section V-E says the array organization provides a peak theoretical bandwidth of ~14.7 TB/s, giving 4.4x headroom over the 3.28 TB/s OS-dataflow demand. These numbers cannot both describe the bandwidth available to the compute cores: the hybrid-bonding interface is listed as 5 TB/s, so a 14.7 TB/s array-side bandwidth cannot be delivered to the systolic arrays. Please clarify the derivation of 14.7 TB/s and distinguish array-internal bandwidth from interface bandwidth. If the interface is 5 TB/s, the true headroom is about 1.5x, and the read-latency sensitivity analysis in Fig. 12 must be re-evaluated under that bound.
  4. [Section IV-C, Eq. (8)] The text states that the uniform prior over S_valid 'yields a worst-case robustness guarantee' against arbitrary future core allocations. This is not correct as written: Eqs. (5)-(8) minimize averages of link utilization and its standard deviation over a uniform distribution. Minimizing an average does not bound the maximum of the quantity. The claim should be reframed as an average-case design choice, or replaced with a true minimax formulation if a worst-case guarantee is intended. The MMLU results provide empirical evidence of robustness but not a worst-case guarantee.
minor comments (6)
  1. [Table I] Llama3 is a dense model, not an MoE model. Its role in the MoE evaluation should be explained, or it should be separated from the MoE model table.
  2. [Section V-C] The notation BFT(2,4,2) is used without definition. Specify what the tuple denotes (e.g., radix, levels, number of parents) so the reader can interpret Solution #3.
  3. [Section V-A] Please describe how the Stratum and H3D-T baselines are instantiated in the same cycle-accurate simulators, so the comparison is clearly iso-methodology. The current text mentions iso-area but not iso-simulator.
  4. [Table VI] The TTFT values for ThAME, Stratum, and TPU v6e are identical by construction, since prefill runs on the same host. State this explicitly in the text to avoid the impression that these are independently measured.
  5. [References] Reference [4] is listed as a DATE 2026 paper; please verify the publication status and year. Some other references (e.g., [20], [21]) also have future-dated volumes; check consistency.
  6. [Reproducibility] No code or simulator configuration is released. Given that all headline results are simulation-based, a reproducibility statement or artifact release would strengthen the paper.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; ThAME's central latency/energy claims are not derived from their own inputs, though the NoC comparison is partly in-sample.

full rationale

The core derivation chain is self-contained rather than circular. The NoC is selected by multi-objective optimization over S_valid (Eqs. 5-10), and some reported latency comparisons (e.g., Fig. 8) are measured over that same S_valid; this is an in-sample evaluation, not a fitted parameter renamed as a prediction. The paper independently validates the NoC on MMLU workload traces (Fig. 10) and through ablations (Fig. 11) that were not used to select the topology, so the central communication claim has external content. The FeFET-NAND capacity discrepancy (Table II's array organization versus the asserted 32 GB per chiplet) is an internal-consistency / feasibility concern, not a circular reduction. Self-citations ([4], [7], [11]) appear only as related work and do not carry the load-bearing argument; the main simulators (SCALE-Sim, NeuroSim, BookSim2, HotSpot, AMOSA) are external or validated against external artifacts. No step equates its output to its input by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on assumed device parameters (FeFET-NAND read latency, 5 TB/s bonding bandwidth, 32 GB/chiplet capacity) and on a self-consistent NoC optimization/evaluation loop. No code or measured silicon is provided. The architecture is a designed integration of known devices, not a new natural entity.

free parameters (4)
  • NoC design Solution #3 (BFT(2,4,2)) = c=4, l=2, m=[4,4], n=[2,2], h_mem=0
    Selected from the Pareto front after MOO over S_valid; all subsequent speedup numbers use this topology. The choice is hand-picked from simulation results and is not independently justified outside the paper.
  • FeFET-NAND read latency = 10 ns (swept 5-100 ns)
    Set from cited FeFET literature; central to the memory-bandwidth argument. Sensitivity shows robustness up to 50 ns, but 10 ns is an assumed device parameter.
  • Hybrid bonding vertical bandwidth / FeFET-NAND capacity = 5 TB/s vertical; 32 GB per chiplet claimed
    Table IV gives 5 TB/s; Section V-F claims 32 GB/chiplet, but Table II array parameters are not shown to yield that capacity. These numbers are load-bearing for the 'all experts on-chip' claim.
  • Compute core configuration = 32 cores, 64x64 systolic array, 800 MHz, 1.13 W
    Chosen under area/thermal budgets (Eqs. 3-4) and used for all end-to-end results; no sensitivity is shown for core count or array size.
assumptions (6)
  • domain assumption FeFET devices provide 5-10 ns read latency, >1e8-1e11 endurance, and thermal stability
    Section IV-B, refs [24],[31],[32]; no measured chiplet-level data for the proposed 3D stack.
  • domain assumption CMOS Directly Bonded to Array (CBA) with Cu-Cu hybrid bonding achieves 5 TB/s without active-area cost
    Section IV-B, refs [33],[43]; assumed feasible at the stated pitch and bandwidth.
  • domain assumption SCALE-Sim, NeuroSim, BookSim2, and HotSpot faithfully model the proposed hardware
    Section V-A; calibrated on different real systems, not on ThAME itself.
  • ad hoc to paper Uniform prior over S_valid gives a worst-case robustness guarantee for NoC design
    Section IV-C; this is a design choice that defines robustness, not an empirical fact about MoE traffic.
  • domain assumption Scheduler solves min-max core allocation via binary search assuming monotone staircase L_i(t_i)
    Section IV-D, Eq. 11; plausible but not experimentally validated.
  • domain assumption MMLU with batch size 32 is representative of real batched MoE serving traffic
    Section V-D; single dataset, single batch size, no sensitivity to other traffic distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts." pith.science (2026). https://pith.science/paper/UYC5UHOX

@misc{pith2026260717074,
  author       = {Pith},
  title        = {Pith review of: ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UYC5UHOX}},
  note         = {Machine review of arXiv:2607.17074}
}
read the original abstract

Mixture of Experts (MoE) architectures have emerged as a dominant paradigm for scaling Large Language Models (LLMs). However, MoE inference on conventional hardware is constrained by three fundamental bottlenecks. These encompass the massive memory bandwidth required to fetch non-contiguous expert weights, the non-deterministic scatter-gather traffic generated by input-dependent token routing, and the tail-latency dependency imposed by synchronous expert output aggregation. To address these challenges, we propose ThAME, a three-dimensional (3D) heterogeneous multi-chiplet architecture for MoE inference. ThAME employs Ferroelectric Field-Effect Transistor (FeFET)-based non-volatile and DRAM-based volatile memory chiplets with a co-designed compute mapping strategy that aligns the distinct computational profiles of attention mechanisms and expert routing. Furthermore, we design a specialized Network-on-Chip communication backbone optimized to mitigate the bottlenecks associated with non-deterministic token routing traffic across the combinatorial space of input-dependent MoE traffic patterns. Experimental results demonstrate that ThAME outperforms state-of-the-art counterparts by up to 15.7x in terms of speedup and improves energy efficiency by up to 9.8x.

Figures

Figures reproduced from arXiv: 2607.17074 by the authors.

Figure 1
Figure 1. High-level architecture of ThAME. The chiplets are connected through the interposer via UCIe. This figure is for illustration purposes only. Hybrid Bonding DRAM FeFET NAND DRAM Host (TPU) PHY Base Die I/O Expert Core PHY FeFET-NAND Chiplet Prefill PHY PHY PHY FeFET NAND Decode Experts Decode Attention Expert Core DRAM Chiplet Base Die I/O Attn Core PHY Attn Core This work was supported by the US National Science Fou… view at source ↗
Figure 2
Figure 2. High-level illustration of a MoE Layer with four experts per layer and top-k (k=2) routing as an example. The model only selects the top k experts with the highest gating probabilities per token within a layer. MoE Add & Norm Add & Norm Multi-Head Attention X Router (Gating Network) E2 E3 Top-k Gating E1 E4 Input Output Softmax Layer i k [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 11
Figure 11. Ablation study showing normalized expert latency and energy across different substrate and NoC configurations considering Qwen1.5 and Deepseek models as representative examples. 0 2 4 6 8 Latency Energy Latency Energy Qwen1.5 Deepseek Normalized Metrics ThAME ThAME-Mesh Stratum-Mesh Stratum [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figures from the paper (2 more)
Figure 11
Figure 11. Figure 11: Ablation study showing normalized expert latency and energy across different substrate and NoC configurations considering Qwen1.5 and Deepseek models as representative examples. 0 2 4 6 8 10 Latency Energy Latency Energy Qwen1.5 Deepseek Normalized Metrics ThAME ThAME…
Figure 12
Figure 12. Figure 12: Sensitivity of expert compute latency to FeFET device read access latency (5 ns to 100 ns). 0.95 1 1.01 1.08 2.2 0 0.5 1 1.5 2 2.5 3 5ns 10ns 20ns 50ns 100ns Normalized Expert Latency FeFET Access Latency ThAME [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 2 linked inside Pith

  1. [1]

    Existing memory hierarchies fail to balance the trade-off between density, leakage, and access latency for these sparse, static weights

    Memory Bottleneck of Experts: The expert layers require massive storage density and high bandwidth to feed compute units. Existing memory hierarchies fail to balance the trade-off between density, leakage, and access latency for these sparse, static weights

  2. [2]

    Consequently, the routing of tokens from their source to the assigned expert computing cores for a particular batch creates a non-deterministic traffic pattern

    Dynamic and Irregular Traffic: Since expert selection is input -dependent, the specific source -destination pairs and traffic volumes cannot be determined prior to inference. Consequently, the routing of tokens from their source to the assigned expert computing cores for a particular batch creates a non-deterministic traffic pattern

  3. [3]

    1) imposes a synchronous gather operation

    Tail Latency: The logical dependency on all selected experts (Eq. 1) imposes a synchronous gather operation . Thus, any network congestion or load imbalance affecting a single expert propagates to the entire batch. Fig. 2: High-level illustration of a MoE Layer with four experts per layer and top-k (k=2) routing as an example. T he model only selects the ...

  4. [4]

    stationary

    Each of the memory chiplets is connected to the host chiplet via die -to-die links. The host chiplet is responsible for facilitating the data exchange among the 3D memory chiplets. Note that there is no direct communication path between the DRAM and FeFET chiplets. The interposer links are facilitated by the Universal Chiplet Interconnect Express (UCIe) s...

  5. [5]

    Design Variables: 𝑑 = (𝑐, 𝑙, 𝒎, 𝒏, ℎ𝑚𝑒𝑚), where ( 𝑐) is the size of the local crossbars (the number of cores per crossbar), (𝑙) is the number of tree levels (see Fig. 4). The vectors (𝒎) and (𝒏) represent the number of children and the number of parents per router assigned at each level of the tree, respectively. The parameter ( ℎ𝑚𝑒𝑚) is the level of the ...

  6. [6]

    Second, the number of links must not exceed that of a standard 2D mesh NoC

    Constraints: First, the NoC must guarantee full host -to- core connectivity: there must be at least one communication path between the host and all cores. Second, the number of links must not exceed that of a standard 2D mesh NoC

  7. [7]

    robustly

    Objectives: Our goal is to “robustly” maximize NoC performance while minimizing area across all combinatorial space of traffic patterns. Conventional NoC frameworks optimize architectures for a deterministic compute mapping [36]. Since the MoE traffic pattern is entirely input-dependent, the traffic pattern is a priori unknown. Our key novelty in the NoC ...

  8. [8]

    CHIME: Chiplet -based Heterogeneous Near -Memory Acceleration for Edge Multimodal LLM Inference,

    Y. Chen et al. , “CHIME: Chiplet -based Heterogeneous Near -Memory Acceleration for Edge Multimodal LLM Inference,” in 2026 Design, Automation & Test in Europe Conference (DATE), IEEE, Apr. 2026

    2026

Show all 58 references
  1. [9]

    A survey of transformers,

    T. Lin et al., “A survey of transformers,” AI Open, vol. 3, 2022

    2022

  2. [10]

    Mixture of Experts in Large Language Models,

    D. Zhang et al., “Mixture of Experts in Large Language Models,” Dec. 22, 2025, arXiv: arXiv:2507.11181. doi: 10.48550/arXiv.2507.11181

    2025 doi

  3. [11]

    A survey of architectural approaches for improving GPGPU performance, programmability and heterogeneity,

    M. Khairy et al. , “A survey of architectural approaches for improving GPGPU performance, programmability and heterogeneity,” Journal of Parallel and Distributed Computing, vol. 127, May 2019

    2019

  4. [12]

    Focus Session: Hardware/Software Co -Design to Accelerate Generative AI Workloads on Heterogeneous Architectures,

    P. Dhingra et al. , “Focus Session: Hardware/Software Co -Design to Accelerate Generative AI Workloads on Heterogeneous Architectures,” in 2026 Design, Automation & Test in Europe Conference (DATE) , IEEE, Apr. 2026

    2026

  5. [13]

    A Survey on Memory -centric Computer Architectures,

    A. Gebregiorgis et al. , “A Survey on Memory -centric Computer Architectures,” J. Emerg. Technol. Comput. Syst., vol. 18, no. 4, Oct. 2022

    2022

  6. [14]

    X-Former: In-Memory Acceleration of Transformers,

    S. Sridharan et al., “X-Former: In-Memory Acceleration of Transformers,” IEEE Trans. VLSI Syst., vol. 31, no. 8, Aug. 2023

    2023

  7. [15]

    HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration,

    P. Dhingra et al., “HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration,” in Proceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design , ACM, Aug. 2024

    2024

  8. [16]

    {DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving,

    Y. Zhong et al., “{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving,” in 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), 2024

    2024

  9. [17]

    H3D-Transformer: A Heterogeneous 3D (H3D) Computing Platform for Transformer Model Acceleration on Edge Devices,

    Y. Luo et al., “H3D-Transformer: A Heterogeneous 3D (H3D) Computing Platform for Transformer Model Acceleration on Edge Devices,” ACM Trans. Des. Autom. Electron. Syst., vol. 29, no. 3, May 2024

    2024

  10. [18]

    Compute -in-Memory Chips for Deep Learning: Recent Trends and Prospects,

    S. Yu et al. , “Compute -in-Memory Chips for Deep Learning: Recent Trends and Prospects,” IEEE Circuits Syst. Mag., vol. 21, no. 3, 2021

    2021

  11. [19]

    Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures,

    P. Dhingra et al., “Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures,” IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., vol. 44, no. 8, Aug. 2025

    2025

  12. [20]

    MixNet: A Runtime Reconfigurable Optical -Electrical Fabric for Distributed Mixture -of-Experts Training,

    X. Liao et al. , “MixNet: A Runtime Reconfigurable Optical -Electrical Fabric for Distributed Mixture -of-Experts Training,” in Proceedings of the ACM SIGCOMM 2025 Conference, ACM, Sept. 2025

    2025

  13. [21]

    Accelerating Distributed MoE Training and Inference with Lina,

    J. Li et al., “Accelerating Distributed MoE Training and Inference with Lina,” Apr. 28, 2024, arXiv: arXiv:2210.17223. doi: 10.48550/arXiv.2210.17223

    2024 doi arXiv

  14. [22]

    A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration,

    W.-H. Huang et al., “A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration,” July 25, 2025, arXiv: arXiv:2507.19142. doi: 10.48550/arXiv.2507.19142

    2025 doi arXiv

  15. [23]

    Stratum: System -Hardware Co -Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving,

    Y. Pan et al. , “Stratum: System -Hardware Co -Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving,” in Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture, ACM, Oct. 2025

    2025

  16. [24]

    Superior QLC Retention (10 Years, 85°C) and Record Memory Window (12.2 V) by Gate Stack Engineering in Ferroelectric FET: from ‘MIFIS’ to

    S.-H. Kuk et al., “Superior QLC Retention (10 Years, 85°C) and Record Memory Window (12.2 V) by Gate Stack Engineering in Ferroelectric FET: from ‘MIFIS’ to ”MIKFIS”,” in 2024 IEEE International Electron Devices Meeting (IEDM), IEEE, Dec. 2024

    2024

  17. [25]

    DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models,

    D. Dai et al., “DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 2024

    2024

  18. [26]

    Newton: A DRAM-maker’s accelerator-in-memory (AiM) architecture for machine learning,

    M. He et al., “Newton: A DRAM-maker’s accelerator-in-memory (AiM) architecture for machine learning,” in 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), IEEE, 2020

    2020

  19. [27]

    Sandisk unveils the future of memory architecture for AI introducing: High bandwidth flash

    Sandisk, “Sandisk unveils the future of memory architecture for AI introducing: High bandwidth flash.” Available: https://documents.sandisk.com/content/dam/asset-library/en_us/assets/public /sandisk/collateral/company/Sandisk-HBF-Fact-Sheet.pdf

  20. [28]

    High Bandwidth Flash: A New Memory for the Next Generation of AI Computing,

    A. Ilkbahar et al., “High Bandwidth Flash: A New Memory for the Next Generation of AI Computing,” Computer, vol. 59, no. 5, May 2026. Fig. 14: Normalized system throughput of ThAME and baseline across varying batch sizes (𝐵 in {1,4,16,64}) for the Deepseek model. 0 5 10 15 20 ...

    2026

  21. [29]

    H3 : H ybrid Architecture Using High Bandwidth Memory and High Bandwidth Flash for Cost-Efficient LLM Inference,

    M. Ha et al., “H3 : H ybrid Architecture Using High Bandwidth Memory and High Bandwidth Flash for Cost-Efficient LLM Inference,” IEEE Comput. Arch. Lett., vol. 25, no. 1, Jan. 2026

    2026

  22. [30]

    3D Memories and Ferroelectrics,

    J. Van Houdt, “3D Memories and Ferroelectrics,” in 2017 IEEE International Memory Workshop (IMW), IEEE, May 2017

    2017

  23. [31]

    A Vertical Channel -All-Around FeFET with Thermally Stable Oxide Semiconductor Achieving High ΔIon > 2µA/cell for 3D Stackable 4F 2 High Speed Memory,

    S. Kabuyanagi et al. , “A Vertical Channel -All-Around FeFET with Thermally Stable Oxide Semiconductor Achieving High ΔIon > 2µA/cell for 3D Stackable 4F 2 High Speed Memory,” in 2024 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits), IEEE, June 2024

    2024

  24. [32]

    The MMLU dataset provides a highly diverse set of multiple -choice questions spanning 57 distinct subjects (ranging from STEM to humanities)

    as an example . The MMLU dataset provides a highly diverse set of multiple -choice questions spanning 57 distinct subjects (ranging from STEM to humanities) . The workload traces for the simulation were generated using PyTorch to capture the non-deterministic MoE traffic. Fig....

  25. [33]

    AttAcc! Unleashing the Power of PIM for Batched Transformer-based Generative Model Inference,

    J. Park et al. , “AttAcc! Unleashing the Power of PIM for Batched Transformer-based Generative Model Inference,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, ACM, Apr. 2024

    2024

  26. [34]

    Cloud TPU v6e

    “Cloud TPU v6e.” Available: https://docs.cloud.google.com/tpu/ docs/v6e

  27. [35]

    In -Datacenter Performance Analysis of a Tensor Processing Unit,

    N. P. Jouppi et al. , “In -Datacenter Performance Analysis of a Tensor Processing Unit,” in Proceedings of the 44th Annual International Symposium on Computer Architecture, ACM, June 2017

    2017

  28. [36]

    Universal Chiplet Interconnect Express (UCIe): An Open Industry Standard for Innovations With Chiplets at Package Level,

    D. Das Sharma et al., “Universal Chiplet Interconnect Express (UCIe): An Open Industry Standard for Innovations With Chiplets at Package Level,” IEEE Trans. Compon., Packag. Manufact. Technol., vol. 12, no. 9, Sept. 2022

    2022

  29. [37]

    3 -D NAND Technology Achievements and Future Scaling Perspectives,

    A. Goda, “3 -D NAND Technology Achievements and Future Scaling Perspectives,” IEEE Trans. Electron Devices, vol. 67, no. 4, Apr. 2020

    2020

  30. [38]

    Unleash Scaling Potential of 3D NAND with Innovative Xtacking® Architecture,

    Z. Huo et al., “Unleash Scaling Potential of 3D NAND with Innovative Xtacking® Architecture,” in 2022 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits), IEEE, June 2022

    2022

  31. [39]

    Ferroelectric Field-Effect Transistor-Based 3-D NAND Architecture for Energy -Efficient on -Chip Training Accelerator,

    W. Shim et al., “Ferroelectric Field-Effect Transistor-Based 3-D NAND Architecture for Energy -Efficient on -Chip Training Accelerator,” IEEE J. Explor. Solid-State Comput. Devices Circuits, vol. 7, no. 1, June 2021

    2021

  32. [40]

    High-Endurance (>1011 cycles) and Thermally-Stable Sub-100nm TiO 2 Channel FeFET for Low -Power Memory Centric 3D -LSI Applications,

    T. Shiokawa et al., “High-Endurance (>1011 cycles) and Thermally-Stable Sub-100nm TiO 2 Channel FeFET for Low -Power Memory Centric 3D -LSI Applications,” in 2023 7th IEEE Electron Devices Technology & Manufacturing Conference (EDTM), IEEE, Mar. 2023

    2023

  33. [41]

    CMOS Directly Bonded to Array (CBA) Technology for Future 3D Flash Memory,

    M. Tagami, “CMOS Directly Bonded to Array (CBA) Technology for Future 3D Flash Memory,” in 2023 International Electron Devices Meeting (IEDM), IEEE, Dec. 2023

    2023

  34. [42]

    A Systematic Methodology for Characterizing Scalability of DNN Accelerators using SCALE -Sim,

    A. Samajdar et al. , “A Systematic Methodology for Characterizing Scalability of DNN Accelerators using SCALE -Sim,” in 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), IEEE, Aug. 2020

    2020

  35. [43]

    On generalized fat trees,

    S. R. Ohring et al. , “On generalized fat trees,” in Proceedings of 9th International Parallel Processing Symposium, IEEE Comput. Soc. Press, 1995

    1995

  36. [44]

    Simba: Scaling Deep -Learning Inference with Multi - Chip-Module-Based Architecture,

    Y. S. Shao et al., “Simba: Scaling Deep -Learning Inference with Multi - Chip-Module-Based Architecture,” in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, ACM, Oct. 2019

    2019

  37. [45]

    Learning -Based Application -Agnostic 3D NoC Design for Heterogeneous Manycore Systems,

    B. K. Joardar et al. , “Learning -Based Application -Agnostic 3D NoC Design for Heterogeneous Manycore Systems,” IEEE Trans. Comput., vol. 68, no. 6, June 2019

    2019

  38. [46]

    A Simulated Annealing -Based Multiobjective Optimization Algorithm: AMOSA,

    S. Bandyopadhyay et al., “A Simulated Annealing -Based Multiobjective Optimization Algorithm: AMOSA,” IEEE Trans. Evol. Computat., vol. 12, no. 3, June 2008

    2008

  39. [47]

    DNN+NeuroSim V2.0: An End -to-End Benchmarking Framework for Compute -in-Memory Accelerators for On -Chip Training,

    X. Peng et al. , “DNN+NeuroSim V2.0: An End -to-End Benchmarking Framework for Compute -in-Memory Accelerators for On -Chip Training,” IEEE Trans. Comput. -Aided Des. Integr. Circuits Syst. , vol. 40, no. 11, Nov. 2021

    2021

  40. [48]

    A detailed and flexible cycle -accurate Network-on-Chip simulator,

    N. Jiang et al., “A detailed and flexible cycle -accurate Network-on-Chip simulator,” in 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), IEEE, Apr. 2013

    2013

  41. [49]

    Hotspot 6.0: Validation, accelacceleration and Extension,

    R. Zhang et al. , “Hotspot 6.0: Validation, accelacceleration and Extension,” University of Virginia, Tech CS-2015-04, Aug. 2015

    2015

  42. [50]

    A Survey of Design and Optimization for Systolic Array - based DNN Accelerators,

    R. Xu et al., “A Survey of Design and Optimization for Systolic Array - based DNN Accelerators,” ACM Comput. Surv., vol. 56, no. 1, Jan. 2024

    2024

  43. [51]

    Ultra High Density SoIC with Sub-micron Bond Pitch,

    Y. H. Chen et al., “Ultra High Density SoIC with Sub-micron Bond Pitch,” in 2020 IEEE 70th Electronic Components and Technology Conference (ECTC), IEEE, June 2020

    2020

  44. [52]

    System on Integrated Chips (SoIC(TM) for 3D Heterogeneous Integration,

    M.-F. Chen et al. , “System on Integrated Chips (SoIC(TM) for 3D Heterogeneous Integration,” in 2019 IEEE 69th Electronic Components and Technology Conference (ECTC), IEEE, May 2019

    2019

  45. [53]

    Efficient Memory Management for Large Language Model Serving with PagedAttention,

    W. Kwon et al. , “Efficient Memory Management for Large Language Model Serving with PagedAttention,” in Proceedings of the 29th Symposium on Operating Systems Principles, ACM, Oct. 2023

    2023

  46. [54]

    [Online]

    meta-llama. [Online]. Available: https://huggingface.co/meta-llama/Meta- Llama-3-8B. [Accessed 25 3 2026]

    2026

  47. [55]

    [Online]

    deepseek -ai. [Online]. Available: https://huggingface.co/deepseek - ai/deepseek-moe-16b-base. [Accessed 25 3 2026]

    2026

  48. [56]

    [Online]

    Qwen. [Online]. Available: https://huggingface.co/Qwen/Qwen3 -30B- A3B. [Accessed 25 3 2026]

    2026

  49. [57]

    [Online]

    allenai. [Online]. Available: https://huggingface.co/allenai/OLMoE -1B- 7B-0125. [Accessed 25 3 2026]

    2026

  50. [58]

    [Online]

    Qwen. [Online]. Available: https://huggingface.co/Qwen/Qwen1.5-MoE- A2.7B. [Accessed 25 3 2026]. Pratyush Dhingra (Graduate Student Member, IEEE) is currently pursuing his Ph.D. degree in Computer Engineering at Washington State University, Pullman, WA, USA. His research inter...

    2026

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.