Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Titanus claims a co-designed accelerator that prunes and quantizes the KV cache during LLM inference, slashing off-chip data movement and boosting energy efficiency by up to 159.9x over an A100 GPU.

desk verdict Plausible on-the-fly KV compression accelerator whose central accuracy claim is unmeasured; the headline speedups rest on an assumption. read the letter →

arxiv 2505.17787 v1 pith:XKWA6QES submitted 2025-05-23 cs.AR

Peilin Chen , Xiaoxuan Yang This is my paper
classification cs.AR
keywords KVcachepruningquantizationcomputing-in-memoryhardwareacceleratorLLMinferencedesignspaceexplorationdatamovement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the key-value cache (KV cache), which stores attention keys and values during LLM inference, is the dominant movable data in long-context decoding, and that an accelerator built for it can compress that cache on the fly. It proposes a cascade pruning-quantization (CPQ) method that first drops low-magnitude KV entries, then quantizes only the survivors to 2-3 bits, and transfers only the non-zero quantized entries plus index/label metadata between chip and off-chip memory. It adds a hierarchical quantization extension (HQE) so each token is quantized only once despite per-channel scale factors that would otherwise require re-quantizing all prior tokens. Built on chiplet computing-in-memory macros that hold all static weights on-chip, the resulting Titanus accelerator claims 159.9x (49.6x) energy efficiency (throughput) over an A100 GPU and 34.8x (29.2x) over FlightLLM on OPT-6.7B. A two-stage design space exploration picks per-layer pruning thresholds and bit-widths.

What carries the argument

The load-bearing mechanism is the cascade pruning-quantization (CPQ) pipeline: a pruning unit (PU) compares every KV element against a per-layer threshold, a quantization unit (QU) and dequantization unit (DQU) apply hierarchical quantization extension (HQE), and only non-zero quantized entries plus a binary index and label cross the off-chip boundary. HQE divides each channel into levels and extends a tolerance range computed in prefill, so a newly generated token is quantized once with a stored scale/zero-point rather than triggering re-quantization of all prior tokens (the 'non-independent per-channel quantization' problem). On the accelerator, digital CIM macros (DCIM) hold the six weight matrices of each OPT decoder layer, a computing engine (CE) skips zero multiplications, and the two-stage DSE (NSGA-II for pruning, progressive bit-width visualization for quantization) fixes the per-layer thresholds and bit-widths.

What would settle it

Run the released Titanus code end-to-end on OPT-6.7B with the DSE-derived per-layer thresholds and bit-widths, measuring accuracy on a held-out benchmark (e.g., WikiText-2 perplexity or MMLU) against the W8A8 baseline; if the full CPQ pipeline's score drops noticeably while pruning-only and quantization-only each claim near-lossless, the central claim that CPQ compresses 'with almost no accuracy loss' fails. Even simpler: record the actual end-to-end accuracy numbers for the combined pipeline, which the paper does not report.

Watch

Extended reading notes

Core claim

Titanus claims that the KV cache does not have to be treated as an opaque tensor shuffled to and from HBM on every decode step. By pruning elements whose absolute value falls below a per-layer threshold and then quantizing only the non-zero survivors with per-channel scales chosen once in prefill, the KV cache movement shrinks by 58.9% on average with almost no accuracy loss (validated on LAMBADA for OPT-125M). HQE makes this practical by extending each channel's tolerance range hierarchically so incoming decode tokens are quantized once, cutting quantization overhead 384.75x at context length 1024. On the hardware side, a chiplet-based digital CIM core keeps all static weights in compute-in-memory macros, the computing engine skips zeros detected from pruning and quantization, and an intra-core pipeline / inter-core parallelism dataflow reduces time-to-first-token. The net claim is that with these pieces, an OPT-6.7B inference accelerator can deliver the reported energy and throughput gains.

Load-bearing premise

The compression is only worth its cost if elements with small absolute values (and the bits dropped by quantization) carry no information that later tokens need; if removing them degrades output quality on data other than the tuning set, the energy and speed gains shrink or vanish.

Editorial extensions

If this is right

  • If the CPQ results hold, future LLM accelerators and GPU runtimes can treat the KV cache as sparse and low-bit without waiting for offline calibration, since pruning and quantization happen on-the-fly during inference.
  • The same CPQ pipeline could scale to longer contexts: since it removes both prefill and newly generated KV entries, the data-movement savings grow as decode length increases, where the KV cache dominates.
  • Reported hardware gains: 159.9x energy efficiency and 49.6x throughput vs A100, 34.8x and 29.2x vs FlightLLM on OPT-6.7B, plus a 107.1x off-chip memory access reduction across the full Titanus design.
  • The two-stage DSE yields a reusable configuration recipe: prune by magnitude, check layer/key/value sensitivity, then assign 2-3 bit widths per layer; this could be adopted as a post-training compression flow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested consequence of the magnitude-threshold pruning rule is that it can be combined with query-aware or recent-token-dense pruning schemes, which might push the compression ratio higher while preserving accuracy.
  • The reported accuracy evidence covers only LAMBADA and TruthfulQA on OPT models; a reader should expect that on broader zero-/few-shot benchmarks the near-lossless claim may need per-dataset threshold re-tuning, since the DSE is tuned on LAMBADA.
  • Because the hardware transfers only non-zero quantized entries, the index/label format is a natural interface for other compression schemes, letting future accelerators swap the pruning/quantization units without changing the dataflow.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes Titanus, a chiplet-based computing-in-memory (CIM) accelerator for LLM inference that performs on-the-fly KV cache pruning and quantization. The main algorithmic contributions are the cascade pruning-quantization (CPQ) method, the hierarchical quantization extension (HQE) strategy to avoid re-quantizing prior tokens, nonzero-only compressed KV transfer, a two-stage design-space exploration (DSE) for per-layer pruning thresholds and bit-widths, and an intra-core pipeline / inter-core parallelism dataflow to reduce time-to-first-token. The authors report RTL synthesis of a 125M-parameter core at 14nm, cycle-accurate simulation, and large claimed improvements over an Nvidia A100 GPU and FlightLLM on OPT-6.7B, with CPQ reducing KV cache movement by 58.9% relative to the baseline.

Significance. If the accuracy of the full CPQ pipeline is confirmed, Titanus would constitute a timely and substantial contribution to KV cache compression and LLM inference acceleration. The strengths of the work include its concrete hardware design (RTL synthesis, cycle-accurate simulator, area/power breakdown), the HQE analysis of quantization-parameter overhead, and public code availability. However, the present manuscript does not supply end-to-end accuracy evidence for the combined pruning-and-quantization pipeline, which is the essential premise for the reported compression and hardware speedups. The practical significance of the headline numbers is therefore not yet established.

major comments (4)
  1. [§5.2, Figure 11] The central claim that CPQ reduces KV cache movement by 58.9% 'with almost no accuracy loss' is not supported by any reported accuracy measurement. Table 1 validates only the HQE quantization step (fp32 to int3) with ROUGE-1 on TruthfulQA; it does not evaluate pruning, it does not evaluate the combined pruning-plus-quantization pipeline, and it does not report perplexity or the W8A8 baseline used in the CPQ experiments. Because the hardware speedups in §5.3 are computed under the assumption that CPQ preserves task quality, an accuracy failure in the combined pipeline would invalidate the practical significance of the headline energy and throughput numbers even if the RTL and cycle-accurate simulation are correct. The authors should report end-to-end accuracy (perplexity and task metrics) for pruning-only and CPQ on the five OPT models, using the exact per-layer thresholds and bit-widths produced by the DSE.
  2. [§3.3, §5.2, Figures 10 and 11] The DSE in Figure 10 selects pruning thresholds and quantization bit-widths on LAMBADA, and the CPQ movement reduction in Figure 11 is evaluated on the same dataset. This selection-on-evaluation setup means the 58.9% movement reduction and the associated 'almost no accuracy loss' claim have not been shown to generalize. The authors should either hold out a portion of LAMBADA for evaluation, evaluate on an independent dataset (e.g., Wikitext or TruthfulQA), or at minimum report the accuracy on a validation split separately from the tuning objective.
  3. [§5.3, Figure 14, Table 2] The headline comparisons with A100 and FlightLLM are based on OPT-6.7B, but the only synthesized core is Titanus-125M (Table 2). The paper states that the five core designs differ mainly in CIM block sizes, yet it does not report area, power, clock, or on-chip buffer parameters for the 6.7B and 13B cores, nor does it explain how the cycle-accurate simulator scales the DCIM macros, SRAM bandwidth, and HBM traffic for these larger models. Without this scaling model, the 159.9x/49.6x and 34.8x/29.2x numbers are extrapolations whose uncertainty is not quantified. Please provide the scaled core estimates and a validation of the simulator against measured RTL behavior at least for one core size.
  4. [§3.1, §5.2, Table 1] There is an unresolved precision mismatch between the hardware pipeline and the algorithm validation. Section 3.1 states that the int8 KV cache enters the pruning and quantization units, while Table 1 evaluates HQE by quantizing fp32 KV cache to int3. The CPQ experiments in Figure 11 use W8A8-quantized OPT models as the baseline. The authors should clarify the input precision of the KV cache in the full CPQ pipeline and evaluate HQE and CPQ from the same int8 W8A8 baseline; otherwise Table 1 does not validate the compression path that the hardware implements.
minor comments (5)
  1. [§3.2] The sentence 'As shown in Figure 3, the breakpoint n0 generates the new level' appears to refer to Figure 2, since Figure 3 is the sensitivity plot; the figure references should be checked.
  2. [§1, §5.1] The abstract and Section 1 use 'this link' for the code repository, while the experimental setup section gives the full URL; the placeholder should be replaced.
  3. [§2.2] The claim that PTQ incurs a 32x overhead for OPT-125M at context length 2048 is stated without derivation; adding the formula used would improve reproducibility.
  4. [Figure 9] The label 'DS' on the horizontal axis is undefined; presumably it means 'decode stage'.
  5. [Table 2] The footnote marker '1' appears after 'DCIM blocks' and then repeats the superscript in the table; the formatting should be cleaned.

Circularity Check

1 steps flagged · score 4.0 of 10

CPQ hyperparameters are tuned on LAMBADA and then evaluated on the same LAMBADA; the 'almost no accuracy loss' claim is in-sample, so the practical speedups rest on an unverified assumption.

  1. fitted input called prediction [Section 5.2 'Benefits of the CPQ Compression Method' (Figure 11), with DSE selection in Section 3.3 (Figure 10).]
    "The evaluation dataset is also LAMBADA. We use quantized W8A8 OPT models as the baseline. In the pruning-only approach, only the binary index and non-zero KV cache are transferred between the accelerator and off-chip memory. The optimal pruning configuration is obtained from the first stage of the DSE in Figure 10. Unlike pruning-only, the CPQ transfers the index, label, and non-zero quantized KV cache, which can further reduce KV cache movement significantly."

    The pruning thresholds and per-layer bit-widths are selected on LAMBADA: Section 3.3 says 'We first leverage the multi-objective optimization algorithm NSGA-II [4] to find the Pareto optimal KV cache pruning configuration. The objectives here are accuracy and average pruning ratio' and Figure 10's experiments use LAMBADA. Figure 11 then reports CPQ's 58.9% KV cache movement reduction 'with almost no accuracy loss' on the same LAMBADA dataset. Because the configuration was chosen to maximize accuracy on LAMBADA, the 'almost no accuracy loss' statement is an in-sample restatement of the optimization objective, not a held-out prediction. No end-to-end accuracy for the combined pruning+quantization pipeline is reported; Table 1 only validates the HQE quantization step on TruthfulQA.

full rationale

Titanus's hardware evaluation is largely self-contained: RTL synthesis, CACTI/NVMain modeling, external CIM macro data from [3], and cycle-accurate simulation based on AccelTran provide independent evidence for the energy/throughput numbers conditional on CPQ preserving accuracy. The two self-citations ([2,34]) are motivational and not load-bearing. The only circular element is in the algorithm evaluation: Section 3.3 uses LAMBADA to select pruning thresholds and per-layer bit-widths (NSGA-II with accuracy and pruning ratio as objectives, plus manual bit-width adjustment), and Section 5.2/Figure 11 reports the resulting 58.9% KV-movement reduction 'with almost no accuracy loss' on the same LAMBADA benchmark. Accuracy on the tuning set is therefore not an independent validation; the combined CPQ pipeline (pruning plus quantization) is never evaluated for end-to-end accuracy on held-out data, since Table 1 only validates the HQE quantization step on TruthfulQA using ROUGE-1. This makes the practical significance of the headline speedups depend on an unquantified in-sample assumption, but the hardware modeling itself is not circular. Score 4.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central performance claims rest on DSE-fitted compression configurations and a simulation stack. No new physical entities are introduced. The paper provides no full-pipeline accuracy measurement, so the free parameters are not independently validated, and the headline gains cannot be separated from the DSE choices and simulator assumptions.

free parameters (4)
  • Layer-wise pruning thresholds for Key and Value = Not listed numerically; found by NSGA-II on LAMBADA
    Thresholds control the 22.3% (pruning-only) and 58.9% (CPQ) movement reductions. Without thresholds, the energy and throughput results cannot be reproduced.
  • Per-layer KV quantization bit-widths = e.g., 3-bit Key and 2-bit Value for OPT-125M after manual adjustment
    Selected by the second DSE stage on LAMBADA; directly determines HQE storage overhead and transferred cache size.
  • Tolerance-range parameters for HQE = Derived from prefill rmax, rmin, s0; number of levels adapts up to 13 per channel
    The level count is data-dependent; the paper reports under 13 levels on average at context 1024, but the exact monitoring thresholds used by the channel monitor are not specified.
  • Operating frequency and process assumptions = 200 MHz at 14nm; CIM macro scaled from 22nm [3]
    All cycle counts and power estimates assume these values; a slower frequency or different CIM characteristics would reduce the headline gains.
assumptions (4)
  • domain assumption Low-magnitude KV elements are unimportant and can be pruned without accuracy loss
    Section 3.1 defines significance by absolute value; no end-to-end accuracy test of the pruned model is provided.
  • domain assumption W8A8 quantization (SmoothQuant) has negligible accuracy loss for the OPT models considered
    Section 5.1 adopts SmoothQuant rather than re-measuring on LAMBADA or TruthfulQA.
  • domain assumption Prefill-stage statistics predict decode-stage KV ranges well enough for hierarchical quantization
    HQE in Sections 3.2 and 4.4 relies on the tolerance range from prefill; evidence is a single OPT-125M out-of-range experiment with max context 64.
  • domain assumption The cycle-accurate simulator with AccelTran, CACTI, NVMain, and 14nm DC synthesis faithfully predicts end-to-end performance for OPT-6.7B and OPT-13B
    The headline comparison in Figure 14 is computed from this stack, not from measurements on the larger cores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration." pith.science (2026). https://pith.science/paper/XKWA6QES

@misc{pith2026250517787,
  author       = {Pith},
  title        = {Pith review of: Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKWA6QES}},
  note         = {Machine review of arXiv:2505.17787}
}
read the original abstract

Large language models (LLMs) have gained great success in various domains. Existing systems cache Key and Value within the attention block to avoid redundant computations. However, the size of key-value cache (KV cache) is unpredictable and can even be tens of times larger than the weights in the long context length scenario. In this work, we propose Titanus, a software-hardware co-design to efficiently compress the KV cache on-the-fly. We first propose the cascade pruning-quantization (CPQ) method to reduce the KV cache movement. The hierarchical quantization extension strategy is introduced to tackle the non-independent per-channel quantization issue. To further reduce KV cache movement, we transfer only the non-zero KV cache between the accelerator and off-chip memory. Moreover, we customize a two-stage design space exploration framework for the CPQ method. A novel pipeline and parallelism dataflow is designed to reduce the first token generation time. Experiments show that Titanus achieves 159.9x (49.6x) and 34.8x (29.2x) energy efficiency (throughput) compared to Nvidia A100 GPU and FlightLLM respectively. The code for Titanus is available at https://github.com/peilin-chen/Titanus-for-LLM-acceleration.

Figures

Figures reproduced from arXiv: 2505.17787 by the authors.

Figure 1
Figure 1. Overview of cascade pruning-quantization method. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Sensitivity of KV cache across different layers to [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Titanus core-level overall architecture. CE and SZ denote the computing engine and scale-zero buffer, respectively. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Computing engine design for dot-product attention [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: On-the-fly pruning unit design. multiplication units (MUs) to enable. Subsequently, the local accu￾mulator aggregates the results from the enabled MUs. The MU can skip zero multiplications using zero detector, and thus reducing energy consumption. The comparator inside…
Figure 8
Figure 8. Figure 8: Intra-core pipeline and inter-core parallelism. [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: Left: Out of range percentage for KV cache across [PITH_FULL_IMAGE:figures/full_fig_p005_9.png]
Figure 10
Figure 10. Figure 10: Two-stage DSE for the CPQ compression method. [PITH_FULL_IMAGE:figures/full_fig_p005_10.png]
Figure 11
Figure 11. Figure 11: KV cache movement reduction using the CPQ. [PITH_FULL_IMAGE:figures/full_fig_p006_11.png]
Figure 14
Figure 14. Figure 14: Energy efficiency and throughput of Titanus over [PITH_FULL_IMAGE:figures/full_fig_p007_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 26 canonical work pages

  1. [1]

    Saleh Ashkboos et al. 2023. Towards end-to-end 4-bit inference on generative large language models. arXiv preprint arXiv:2310.09259 (2023)

    arXiv 2023

  2. [2]

    Peilin Chen et al. 2025. Optimizing and exploring system performance in compact processing-in-memory-based chips. arXiv preprint arXiv:2502.21259 (2025)

    work page Pith review arXiv 2025

  3. [3]

    Yu-Der Chih et al. 2021. 16.4 An 89TOPS/W and 16.3 TOPS/mm 2 all-digital SRAM-based full-precision compute-in memory macro in 22nm for machine- learning edge applications. In ISSCC, Vol. 64. IEEE, 252–254

    work page 2021

  4. [4]

    Kalyanmoy Deb et al. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE TEVC 6, 2 (2002), 182–197

    work page 2002

  5. [5]

    Tim Dettmers et al. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. NeurIPS 35 (2022), 30318–30332

    work page 2022

  6. [6]

    Qichen Fu et al. 2024. Lazyllm: Dynamic token pruning for efficient long context llm inference. arXiv preprint arXiv:2407.14057 (2024)

    arXiv 2024

  7. [7]

    Tae Jun Ham et al . 2020. Aˆ 3: Accelerating attention mechanisms in neural networks with approximation. In HPCA. IEEE, 328–341

    work page 2020

  8. [8]

    Tae Jun Ham et al . 2021. ELSA: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks. In ISCA. IEEE, 692– 705

    work page 2021

Show all 37 references
  1. [9]

    Coleman Hooper et al. 2024. Kvquant: Towards 10 million context length llm inference with kv cache quantization. arXiv preprint arXiv:2401.18079 (2024)

    2024 arXiv

  2. [10]

    Maxwell Horton et al. 2024. KV Prediction for Improved Time to First Token. arXiv preprint arXiv:2410.08391 (2024)

    2024 arXiv

  3. [11]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81

    2004

  4. [12]

    Ji Lin et al. 2024. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. MLSys 6 (2024), 87–100

    2024

  5. [13]

    Stephanie Lin et al . 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 (2021)

    2021 arXiv

  6. [14]

    Zirui Liu et al. 2024. Kivi: A tuning-free asymmetric 2bit quantization for kv cache. arXiv preprint arXiv:2402.02750 (2024)

    2024 arXiv

  7. [15]

    Hongyin Luo et al. 2024. Addition is all you need for energy-efficient language models. arXiv preprint arXiv:2410.00907 (2024)

    2024 arXiv

  8. [16]

    Naveen Muralimanohar et al. 2009. CACTI 6.0: A tool to model large caches. HP laboratories 27 (2009), 28

    2009

  9. [17]

    Denis Paperno et al. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context. arXiv preprint arXiv:1606.06031 (2016)

    2016 arXiv

  10. [18]

    Matthew Poremba et al. 2015. Nvmain 2.0: A user-friendly memory simulator to model (non-) volatile memory systems. IEEE CAL 14, 2 (2015), 140–143

    2015

  11. [19]

    Zheng Qu et al. 2022. Dota: detect and omit weak attentions for scalable trans- former acceleration. In ASPLOS. 14–26

    2022

  12. [20]

    Wenqi Shao et al. 2023. Omniquant: Omnidirectionally calibrated quantization for large language models. arXiv preprint arXiv:2308.13137 (2023)

    2023 arXiv

  13. [21]

    Ying Sheng et al. 2023. Flexgen: High-throughput generative inference of large language models with a single gpu. In ICML. PMLR, 31094–31116

    2023

  14. [22]

    Linghao Song et al . 2017. Pipelayer: A pipelined reram-based accelerator for deep learning. In HPCA. IEEE, 541–552

    2017

  15. [23]

    Fengbin Tu et al . 2022. A 28nm 15.59 𝜇J/token full-digital bitline-transpose CIM-based sparse transformer accelerator with pipeline/parallel reconfigurable modes. In ISSCC, Vol. 65. IEEE, 466–468

    2022

  16. [24]

    Fengbin Tu et al. 2022. A 28nm 29.2 TFLOPS/W BF16 and 36.5 TOPS/W INT8 reconfigurable digital CIM processor with unified FP/INT pipeline and bitwise in-memory booth multiplication for cloud deep learning acceleration. In ISSCC, Vol. 65. IEEE, 1–3

    2022

  17. [25]

    Shikhar Tuli et al. 2023. AccelTran: A sparsity-aware accelerator for dynamic inference with transformers. IEEE TCAD 42, 11 (2023), 4038–4051

    2023

  18. [26]

    Naveen Verma et al. 2019. In-memory computing: Advances and prospects. IEEE SSC-M 11, 3 (2019), 43–55

    2019

  19. [27]

    Hanrui Wang et al. 2021. Spatten: Efficient sparse attention architecture with cascade token and head pruning. In HPCA. IEEE, 97–110

    2021

  20. [28]

    Yizhi Wang et al . 2017. An energy-efficient architecture for binary weight convolutional neural networks. TVLSI 26, 2 (2017), 280–293

    2017

  21. [29]

    Guangxuan Xiao et al. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In ICML. PMLR, 38087–38099

    2023

  22. [30]

    Yuhui Xu et al. 2024. Think: Thinner key cache by query-driven pruning. arXiv preprint arXiv:2407.21018 (2024)

    2024 arXiv

  23. [31]

    Huanrui Yang et al . 2022. Hero: Hessian-enhanced robust optimization for unifying and improving generalization and quantization performance. In DAC. 25–30

    2022

  24. [32]

    Jingfeng Yang et al. 2024. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data 18, 6 (2024), 1–32

    2024

  25. [33]

    Xiaoxuan Yang et al. 2020. ReTransformer: ReRAM-based processing-in-memory architecture for transformer acceleration. In ICCAD. 1–9

    2020

  26. [34]

    Xiaoxuan Yang et al. 2022. Research progress on memristor: From synapses to computing systems. IEEE TCAS-I 69, 5 (2022), 1845–1857

    2022

  27. [35]

    Shulin Zeng et al. 2024. Flightllm: Efficient large language model inference with a complete mapping flow on fpgas. In FPGA. 223–234

    2024

  28. [36]

    Haopeng Zhang et al. 2024. A systematic survey of text summarization: From statistical methods to large language models. Comput. Surveys (2024)

    2024

  29. [37]

    Susan Zhang et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022)

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.