Pith. sign in

REVIEW 9 cited by

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.12065 v2 pith:AHPKUNJ7 submitted 2024-02-19 cs.LG cs.AIcs.CL

Yuxuan Yue , Zhihang Yuan , Haojie Duanmu , Sifan Zhou , Jianlong Wu , Liqiang Nie This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords quantizationllmscachememorywkvquantchallengeslanguagelarge
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) face significant deployment challenges due to their substantial memory requirements and the computational demands of auto-regressive text generation process. This paper addresses these challenges by focusing on the quantization of LLMs, a technique that reduces memory consumption by converting model parameters and activations into low-bit integers. We critically analyze the existing quantization approaches, identifying their limitations in balancing the accuracy and efficiency of the quantized LLMs. To advance beyond these limitations, we propose WKVQuant, a PTQ framework especially designed for quantizing weights and the key/value (KV) cache of LLMs. Specifically, we incorporates past-only quantization to improve the computation of attention. Additionally, we introduce two-dimensional quantization strategy to handle the distribution of KV cache, along with a cross-block reconstruction regularization for parameter optimization. Experiments show that WKVQuant achieves almost comparable memory savings to weight-activation quantization, while also approaching the performance of weight-only quantization.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.

  2. OpenGCRAM: An Open-Source Gain Cell Compiler Enabling Design-Space Exploration for AI Workloads

    cs.AR 2025-07 conditional novelty 7.0 of 10

    OpenGCRAM automatically generates layout-ready GCRAM memory banks and simulates their area, speed, and power, enabling designers to explore GCRAM configurations for AI workloads.

  3. PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A multi-agent repair framework that samples multiple edit locations and iteratively reflects on patch attempts reaches 76.0% Pass@1 on SWE-bench-Verified, up to a 7.8% relative gain over SWE-agent.

  4. Harmonia: Algorithm-Hardware Co-Design for Memory- and Compute-Efficient BFP-based LLM Inference

    cs.AR 2026-02 conditional novelty 6.0 of 10

    Harmonia runs LLM inference with all activations in block floating point (BFP) and a 4-bit KV cache, reporting 3.08x average speedup, 2.03x energy savings, and under 1% accuracy loss on LongBench.

  5. Which Heads Matter for Reasoning? RL-Guided KV Cache Compression

    cs.CL 2025-10 conditional novelty 6.0 of 10

    A small set of "reasoning heads" found by RL can keep full KV cache while other heads are compressed to a constant size, giving 20–50% cache savings with near-lossless accuracy.

  6. PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling

    cs.LG 2025-06 conditional novelty 6.0 of 10

    PCDVQ compresses LLM weights to 2 bits by quantizing vector directions and magnitudes separately with distribution-matched codebooks, reporting modest zero-shot accuracy gains over prior vector quantization baselines.

  7. PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs

    cs.CL 2025-05 conditional novelty 6.0 of 10

    PM-KVQ introduces progressive bit-width shrinking, per-block memory allocation, and positional-interpolation calibration to make 2-bit KV cache quantization nearly lossless on long-CoT LLMs.

  8. Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.

  9. Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents

    cs.AI 2025-02 conditional novelty 5.0 of 10

    The authors propose episodic memory, with five defining properties, as the unifying framework needed for LLM agents to learn and remember over long time horizons.

Pith tools