Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storage

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hierarchical block-extraction and delta-indexed storage scheme claims up to 6.44x faster SpMV for sparse LLM weights.

desk verdict Solid extraction/format idea, but the central speedup claims are unverifiable as submitted because the kernel pseudocode is broken and no code or data is released. read the letter →

arxiv 2507.12205 v1 pith:PGIGHK2P submitted 2025-07-16 cs.DC

Junqing Lin , Jingwei Sun , Mingge Lu , Guangzhong Sun This is my paper
classification cs.DC
keywords sparsematrix-vectormultiplicationLLMinferenceGPUkernelsblockextractiondeltaindexingcompressedrowgreedymatchingweightmatrixsparsity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that sparse matrix-vector multiplication, the bottleneck operation when a pruned large language model generates text one token at a time, can be made faster and more memory-efficient by exploiting the structure of LLM weight matrices instead of using formats designed for scientific computing. It proposes a two-phase GPU method: an offline phase that repeatedly extracts blocks of non-zero entries at several granularities, and a compressed sparse format that stores column positions as small delta indices. The paper reports speedups up to 6.44x over state-of-the-art SpMV libraries and up to 55.4% storage reduction versus the standard compressed sparse row format, on real pruned weight matrices. If the results hold, local single-request LLM decoding on consumer GPUs would run faster and fit larger models in memory.

What carries the argument

The machinery has two parts. First, EC-SpMV's hierarchical block extraction greedily pairs rows by the number of shared non-zero columns, extracts the shared columns as a two-row block, blanks those entries, and repeats on the residue; the extracted blocks are then encoded as a new, smaller sparse matrix so the next level can build blocks of twice as many rows. Second, EC-CSR stores the resulting block sets in five arrays, namely row_indices, block_indptr, base_indices, delta_indices, and block_values, with each thread keeping one base column index and a run of small deltas that are summed cumulatively to recover absolute positions. Padding and permutation of the delta and value arrays force vectorized and coalesced GPU memory access, which is what converts the format's compactness into kernel speed.

What would settle it

Take a weight matrix at 80% sparsity whose non-zero column positions are chosen uniformly at random rather than clustered, run the preprocessing, and count how many delta indices exceed 64; if that fraction is large enough to force wider deltas or heavy padding, the storage savings and speedup over the regular sparse-row format will largely disappear.

Watch

Extended reading notes

Core claim

The central claim is that the non-zero patterns of pruned LLM weight matrices contain hierarchical block structure that can be captured by repeated rounds of row-pairing and block extraction at multiple granularities, and that once blocks are extracted, column indices inside each block cluster tightly enough that delta (difference-based) indexing with 4- or 8-bit integers can replace absolute 16- or 32-bit indices. This combination improves data locality, because blocks amortize index and input-vector access across several rows, and cuts storage, because each stored position is small. The paper reports kernel speedups up to 4.36x in FP32 and up to 6.44x in FP16 over the strongest baselines, geometric mean speedups of 1.38x and 2.43x respectively, and storage reductions up to 55.4% relative to the standard compressed sparse row format.

Load-bearing premise

The load-bearing premise is that after row reordering and block extraction, the non-zero entries of a pruned LLM weight matrix sit close enough together in column space that almost all neighboring-position differences fit in 4 or 8 bits; the paper demonstrates that clustering only for one pruning method and one pair of model families at 70-90% sparsity.

Editorial extensions

If this is right

  • In the single-request decoding regime, pruned models can generate tokens noticeably faster: the paper's end-to-end test reports 10.8-15.1% throughput gains at three prediction lengths.
  • The storage cuts are large enough to change which GPU a model fits on: a 70% sparse 7B-class model that occupies 13.48 GB in dense form drops to 6.28 GB in the new format.
  • The once-per-deployment preprocessing cost, under 100 seconds for most matrices in the paper, does not add per-token latency and is amortized over use.
  • Using 4-bit deltas at high sparsity can introduce substantial padding overhead, reaching 30.43% at 90% sparsity, so the results support 8-bit deltas as the safer configuration.
  • The same block-extraction and compressed-storage ideas are pointed by the paper toward other sparse operators, namely sparse matrix-matrix multiplication and sampled dense-dense matrix multiplication.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the locality that makes delta indexing work is a general property of pruned LLM weights, then pruning methods could be redesigned to explicitly cap the distance between adjacent non-zeros, which would make 4-bit deltas viable at higher sparsity; the paper itself points toward this co-design idea.
  • The speedups are measured in the single-request decoding regime with batch size one; at larger batches the operation becomes more like a dense GEMM bottleneck, so the kernel-level gains should shrink away from that regime.
  • A direct stress test would be to run the same pipeline on matrices pruned with a different, less structure-preserving scheme; if the delta-outlier rate jumps, padding overhead will eat the storage and speed advantages.
  • The hierarchical extraction is greedy and approximate because optimal block selection is NP-hard; a better matching heuristic or an exact solver on smaller matrices could reveal how much block coverage is being left on the table.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes EC-SpMV, a two-phase approach for SpMV in sparse LLM weight matrices. The offline phase performs hierarchical block extraction that greedily forms blocks at multiple granularities through multi-round row matching, then stores the extracted blocks in a new format, EC-CSR, which uses low-precision delta indices plus base indices to compress storage. The online phase is a GPU kernel that assigns each warp to one block and uses vectorized, coalesced memory accesses. The authors evaluate EC-SpMV on SparseGPT-pruned LLaMA and OPT matrices at 70--90% sparsity on three GPUs, reporting up to 6.44x speedup over selected baselines and up to 55.4% storage reduction relative to CSR, and they include an end-to-end llama.cpp case study.

Significance. The problem is practically important: decoding-phase GEMV on pruned LLMs is a real bottleneck for local inference, and storage reduction is directly useful for fitting models on consumer GPUs. The hierarchical block-extraction idea is a reasonable extension of aggressive tiling to LLM sparsity, and the delta-index compression is a concrete, cheap mechanism for reducing both storage and memory traffic. If the empirical claims hold, the contribution is useful for practitioners, though the algorithmic novelty is incremental. The paper has the right kind of target matrices and multiple GPU configurations. However, the current evidence is not verifiable: the kernel listing is internally inconsistent, no code or data are released, and the benchmark setup has several confounds. The central idea is defensible, but the manuscript needs substantial revision before the claims can be accepted.

major comments (4)
  1. [Section 7, Listing 1] Listing 1 is the only specification of the kernel, but as written it is not a working kernel. Line 44 reads x_values[current_index] although current_index is never assigned; the only index variable is base_index, so this is either an undefined reference or an unreported rename. Line 11 computes warp_id_global = blockIdx.x * (gridDim.x/warp_size) + warp_id, but the correct per-block warp stride is blockDim.x / warp_size; using gridDim.x makes warp IDs collide across blocks, causing races. The listing also does not implement the per-thread sub-block structure described in Section 6.2: base_indices[0], delta_indices[0], and block_values[0] bypass the required lane-specific indexing. Finally, after warpReduceSum, the atomicAdd on lines 59--62 writes res[j] instead of the reduced results[j]. These are not cosmetic errors: because no source code or benchmark data are released, the listing is the only implementation specification, and it cannot produce the reported speedups. The authors should release the exact CUDA kernel and data, or repair the listing so that every array access and index computation matches the described algorithm.
  2. [Section 8.1] The central claims are empirical benchmark comparisons, but the paper reports only single-number geometric means and 'up to' maxima. There are no error bars, no number of repeated runs, and no per-matrix result tables, so the reader cannot assess whether the 1.38x and 2.43x geometric-mean speedups are stable or dominated by noise. The authors should report the distribution of per-matrix speedups, the number of trials, and the variance, and should identify the exact matrix and configuration that produce the headline 6.44x speedup.
  3. [Section 8.1 (experimental setup)] TileSpMV was tested with CUDA Toolkit 11.1 while all other baselines and EC-SpMV used version 12.2. Unequal compiler and runtime versions can materially change SpMV performance, and TileSpMV is one of the strongest baselines at 90% sparsity in Figure 7. The authors should rebuild TileSpMV with the same CUDA version as the other baselines or provide a quantitative justification for the version difference.
  4. [Section 8.1 (baseline configuration)] AlphaSparse's search time is capped at 10,000 seconds, which can prevent the search from converging to its best kernel for these matrices. Since AlphaSparse is the best FP32 baseline, the paper should report the actual search times, the distribution of resulting kernels, and a sensitivity analysis of the cap. In addition, Flash-LLM [40], which is cited as a state-of-the-art sparse LLM inference system, is not benchmarked anywhere in the evaluation; if it is out of scope, this should be stated explicitly, otherwise the 'state-of-the-art' claim is incomplete.
minor comments (5)
  1. [Section 6.2] The text contains duplicated and ungrammatical sentences ('Moreover, a coarse-grained block comprises multiple finer-grained units. Moreover, a coarse-grained block are composed of multiple finer-grained units.') and 'an base index'; these should be corrected.
  2. [Section 8.1] The abstract and introduction quote 'up to 6.44x' and 'up to 55.4%' as headline numbers, but the figures and text report only geometric means; the paper should identify the specific matrix and hardware configuration that achieve these maxima.
  3. [Section 6.2] Figure 5 would be more informative with per-model and per-layer breakdowns and error bars; as presented, it is not clear how many matrices contribute to each cumulative distribution or whether the pattern is stable across layers.
  4. [Section 3 and Section 6.2] The free parameters delta precision R_P, vector_size, and the block-clipping threshold are not accompanied by a sensitivity study; since the storage and speedup claims depend on these choices, reporting the selected values and their robustness would strengthen the paper.
  5. [Section 8.4] The end-to-end case study reports a perplexity increase from 5.12 to 24.00 after 70% SparseGPT pruning, which is a severe quality degradation; the paper should discuss this accuracy-efficiency tradeoff so that the speedup numbers are not presented without context.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EC-SpMV's speedup and storage claims are empirical benchmark results against external libraries and measured format costs, not derivations that reduce to their own inputs.

full rationale

The paper's central claims are empirical: kernel speedups are measured against cuSPARSE, CSR5, TileSpMV, AlphaSparse, DASP, and cuBLAS on three GPUs, and storage reductions are computed against CSR on real SparseGPT-pruned LLaMA/OPT matrices. These are externally falsifiable benchmarks, not quantities forced by the paper's definitions. The delta-index design in Section 6.2 is motivated by measured distributions of delta indices, and the reported storage savings explicitly account for padding overhead (Table 2); notably, EC-CSR-4 at 90% sparsity gives only a modest or sometimes negative improvement, which shows the result is not guaranteed by construction. The hierarchical block extraction is a greedy heuristic with stated complexity, and load balancing is an empirical optimization. The only apparent self-reference is reference [21], a prior Lo-SpMM paper by overlapping authors, cited in the related-work discussion of tiling; it is not load-bearing for any uniqueness claim or for the EC-SpMV design. There is no imported uniqueness theorem, no ansatz smuggled via self-citation, and no renaming of a known result. The kernel listing's undefined variable and warp-id arithmetic are implementation-correctness and reproducibility concerns, not circularity: they do not make the evaluation a restatement of the method's assumptions.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims are empirical and engineering-oriented. The main free choices are index precision, vectorization width, and clipping threshold. The key domain assumption is that pruned LLM weights have moderate, uniform sparsity with locally clustered indices. EC-CSR is a data structure, not a new physical entity, so no invented entities are introduced.

free parameters (3)
  • delta precision R_P = 4 or 8 bits
    Chosen based on the cumulative distribution of delta indices in Llama2 pruned matrices (Figure 5). At 90% sparsity, uint4 padding overhead reaches 30.43%, showing the choice trades storage against padding.
  • vector_size = 4 for most blocks, 1 for 1-grained blocks
    Set to optimize vectorized memory access on NVIDIA GPUs; not derived from first principles and directly affects kernel performance.
  • block clipping threshold = not specified
    Described as dynamically adjusted in Section 5, but the adjustment rule and values are not given, so load balancing results are not fully reproducible from the paper.
assumptions (3)
  • domain assumption Sparse LLM weight matrices have moderate sparsity and a relatively uniform distribution of non-zero elements.
    Sections 2.2 and 6.2 rely on this to justify delta indexing and block extraction. Figure 5 supports it for SparseGPT-pruned LLaMA and OPT matrices, but not for all pruning methods.
  • standard math Optimal block extraction is NP-hard, so a greedy matching heuristic is sufficient.
    Section 4.3 invokes maximum-weight matching NP-hardness and adopts greedy row matching with O(n^2 log n) complexity.
  • domain assumption GPU execution model assumptions about warps, coalescing, and vectorized loads improve performance as described.
    Sections 5 and 6.3 design the kernel around warp_size and vector_size behavior on NVIDIA GPUs; these are hardware-specific assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storage." pith.science (2026). https://pith.science/paper/PGIGHK2P

@misc{pith2026250712205,
  author       = {Pith},
  title        = {Pith review of: Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storage},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGIGHK2P}},
  note         = {Machine review of arXiv:2507.12205}
}
read the original abstract

Sparse Matrix-Vector Multiplication (SpMV) has become a critical performance bottleneck in the local deployment of sparse Large Language Models (LLMs), where inference predominantly operates on workloads during the decoder phase with a batch size of one. Existing SpMV kernels and sparse matrix formats, originally designed for scientific computing, fail to exploit the unique structure patterns inherent in sparse LLMs, resulting in suboptimal performance and excessive storage overhead. This paper presents EC-SpMV, a GPU-optimized SpMV approach for accelerating sparse LLM inference. EC-SpMV introduces (1) a hierarchical block extraction algorithm that captures multiple granularities of block structures within sparse LLMs, and (2) a novel compressed sparse format (EC-CSR) that employs delta indexing to reduce storage overhead and enhance memory access efficiency. Evaluated on real sparse weight matrices from LLaMA and OPT models, EC-SpMV achieves up to 6.44x speedup over state-of-the-art SpMV libraries and reduces storage overhead by up to 55.4% compared to CSR.

Figures

Figures reproduced from arXiv: 2507.12205 by the authors.

Figure 1
Figure 1. An illustration of aggressive tiling: blue and brown [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of EC-SpMV. To overcome these limitations, we propose a hierarchical block extraction method that supports adaptive and multi-granularity block identification. EC-SpMV performs multi-level aggregation, where each level generates coarser-grained blocks from finer-grained ones extracted at the previous level. This hierarchical strategy en￾sures that fine-grained elements are progressively integrated into as l… view at source ↗
Figure 3
Figure 3. An illustration of hierarchical block extraction. Each non-zero element in a sparse matrix represents a dense column [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: An example of EC-CSR storing three block sets. As [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Cumulative distribution of delta indices in sparse [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Illustration of permutation. Assume warp_size is 2. warp. If each subblock is stored contiguously, as shown on the left of [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The relative performance of various SpMV kernel implementations compared to cuBLAS. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The preprocess overhead of EC-SpMV. The preprocessing procedure of EC-SpMV comprises two main stages: hierarchical block extraction and sparse matrix format con￾version. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: The relative storage overhead on different formats. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: illustrates the contributions of various optimizations im￾plemented in EC-SpMV to SpMV kernel performance, taking FP32 precision on the NVIDIA 3080 Ti as an example. Among these, EC-SpMV with only index compression improves memory access efficiency by replacing absolu…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 39 canonical work pages

  1. [40]

    Xia, H., Zheng, Z., Li, Y., Zhuang, D., Zhou, Z., Qiu, X., Li, Y., Lin, W., and Song, S. L. Flash-llm: Enabling cost-effective and highly-efficient large generative model inference with unstructured sparsity. Proceedings of the VLDB Endowment 17, 2 (2023), 211–224

    work page 2023

  2. [1]

    https://docs.nvidia.com/cuda/cublas/index

    Basic linear algebra on nvidia gpus. https://docs.nvidia.com/cuda/cublas/index. html, 2024

    work page 2024

  3. [2]

    M., Buluç, A., Williams, S., and Y ang, C.Optimizing sparse matrix- multiple vectors multiplication for nuclear configuration interaction calculations

    Aktulga, H. M., Buluç, A., Williams, S., and Y ang, C.Optimizing sparse matrix- multiple vectors multiplication for nuclear configuration interaction calculations. In 2014 IEEE 28th International Parallel and Distributed Processing Symposium (2014), IEEE, pp. 1213–1222

    work page 2014

  4. [3]

    Fast sparse matrix-vector multiplication on gpus for graph applications

    Ashari, A., Sedaghati, N., Eisenlohr, J., Parthasarath, S., and Sadayappan, P. Fast sparse matrix-vector multiplication on gpus for graph applications. In SC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2014), IEEE, pp. 781–792

    work page 2014

  5. [4]

    Efficient sparse matrix-vector multiplication on cuda

    Bell, N., and Garland, M. Efficient sparse matrix-vector multiplication on cuda. Tech. rep., 2008

    work page 2008

  6. [5]

    On the relations between ilus and factored approx- imate inverses

    Bollhöfer, M., and Saad, Y. On the relations between ilus and factored approx- imate inverses. SIAM Journal on Matrix Analysis and Applications 24 , 1 (2002), 219–237

    work page 2002

  7. [6]

    In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (2022), IEEE, pp

    Du, Z., Li, J., W ang, Y., Li, X., Tan, G., and Sun, N.Alphasparse: Generating high performance spmv codes directly from sparse matrices. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (2022), IEEE, pp. 1–15

    work page 2022

  8. [7]

    Scaling algorithms for weighted matching in general graphs

    Duan, R., Pettie, S., and Su, H.-H. Scaling algorithms for weighted matching in general graphs. ACM Transactions on Algorithms (TALG) 14 , 1 (2018), 1–35

    work page 2018

Show all 47 references
  1. [8]

    Spinfer: Leveraging low-level sparsity for efficient large language model inference on gpus

    Fan, R., Yu, X., Dong, P., Li, Z., Gong, G., Wang, Q., Wang, W., and Chu, X. Spinfer: Leveraging low-level sparsity for efficient large language model inference on gpus. In Proceedings of the Twentieth European Conference on Computer Systems (2025), pp. 243–260

    2025

  2. [9]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Frantar, E., and Alistarh, D. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning (2023), PMLR, pp. 10323–10337

    2023

  3. [10]

    Leveraging index compression techniques to optimize the use of co-processors

    Freire, M., Marichal, R., Martinez, A., Padron, D., Dufrechou, E., and Ezzatti, P. Leveraging index compression techniques to optimize the use of co-processors. Journal of Computer Science and Technology 24 , 1 (2024), e01–e01

    2024

  4. [11]

    Sparse GPU kernels for deep learning

    Gale, T., Zaharia, M., Young, C., and Elsen, E. Sparse GPU kernels for deep learning. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020, Virtual Event / Atlanta, Georgia, USA, November 9-19, 2020 (2020), IE...

    2020

  5. [12]

    ggerganov/llama

    Gerganov, G. ggerganov/llama. cpp: Port of facebook’s llama model in c/c++, 2023

    2023

  6. [13]

    L., and Daga, M

    Greathouse, J. L., and Daga, M. Efficient sparse matrix-vector multiplication on gpus using the csr storage format. In SC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2014), IEEE, pp. 769–780

    2014

  7. [14]

    Y., Leng, J., Qiu, Y., Guan, Y., W ang, Z., Jia, X., Li, X., Guo, M., and Zhu, Y

    Guo, C., Hsueh, B. Y., Leng, J., Qiu, Y., Guan, Y., W ang, Z., Jia, X., Li, X., Guo, M., and Zhu, Y. Accelerating sparse dnn models without hardware-support via tile-wise sparsity. In SC20: International Conference for High Performance Computing, Networking, Storage and Analys...

    2020

  8. [15]

    In Proceedings of the 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2019, Washington, DC, USA, February 16-20, 2019 (2019), ACM, pp

    Hong, C., Sukumaran-Rajam, A., Nisa, I., Singh, K., and Sadayappan, P.Adap- tive sparse tiling for sparse matrix multiplication. In Proceedings of the 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2019, Washington, DC, USA, February 16-20...

    2019

  9. [16]

    Flashdecoding++: Faster large language model inference with asynchronization, flat gemm optimization, and heuristics

    Hong, K., Dai, G., Xu, J., Mao, Q., Li, X., Liu, J., Dong, Y., Wang, Y., et al. Flashdecoding++: Faster large language model inference with asynchronization, flat gemm optimization, and heuristics. Proceedings of Machine Learning and Systems 6 (2024), 148–161

    2024

  10. [17]

    In Proceedings of the 25th ACM SIGPLAN symposium on principles and practice of parallel program- ming (2020), pp

    Jiang, P., Hong, C., and Agrawal, G.A novel data transformation and execution strategy for accelerating sparse matrix multiplication on gpus. In Proceedings of the 25th ACM SIGPLAN symposium on principles and practice of parallel program- ming (2020), pp. 376–388

    2020

  11. [18]

    Maximum bounded 3-dimensional matching is max snp-complete

    Kann, V. Maximum bounded 3-dimensional matching is max snp-complete. Information Processing Letters 37 , 1 (1991), 27–35

    1991

  12. [19]

    Computational complexity of the perfect matching problem in hypergraphs with subcritical density

    Karpiński, M., Ruciński, A., and Szymańska, E. Computational complexity of the perfect matching problem in hypergraphs with subcritical density. Interna- tional Journal of Foundations of Computer Science 21 , 06 (2010), 905–924

    2010

  13. [20]

    Optimizing sparse matrix-vector multiplication using index and value compression

    Kourtis, K., Goumas, G., and Koziris, N. Optimizing sparse matrix-vector multiplication using index and value compression. In Proceedings of the 5th conference on Computing frontiers (2008), pp. 87–96

    2008

  14. [21]

    ACM Transactions on Architecture and Code Optimization (2024)

    Lin, J., Sun, J., Shi, X., Zhang, H., Yu, X., W ang, X., Y ao, J., and Sun, G.Lo-spmm: Low-cost search for high-performance spmm kernels on gpus. ACM Transactions on Architecture and Code Optimization (2024)

    2024

  15. [22]

    Csr5: An efficient storage format for cross-platform sparse matrix-vector multiplication

    Liu, W., and Vinter, B. Csr5: An efficient storage format for cross-platform sparse matrix-vector multiplication. In Proceedings of the 29th ACM on Interna- tional Conference on Supercomputing (2015), pp. 339–350

    2015

  16. [23]

    Spp: Sparsity-preserved parameter-efficient fine-tuning for large language models, 2024

    Lu, X., Zhou, A., Xu, Y., Zhang, R., Gao, P., and Li, H. Spp: Sparsity-preserved parameter-efficient fine-tuning for large language models, 2024

    2024

  17. [24]

    Dasp: Specific dense matrix multiply-accumulate units accelerated general sparse matrix-vector multiplication

    Lu, Y., and Liu, W. Dasp: Specific dense matrix multiply-accumulate units accelerated general sparse matrix-vector multiplication. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2023), pp. 1–14

    2023

  18. [25]

    Llm-rec: Personalized recommendation via prompting large language models

    Lyu, H., Jiang, S., Zeng, H., Xia, Y., Wang, Q., Zhang, S., Chen, R., Leung, C., Tang, J., and Luo, J. Llm-rec: Personalized recommendation via prompting large language models. In Findings of the Association for Computational Linguistics: NAACL 2024 (2024), pp. 583–612

    2024

  19. [26]

    Advances in neural information processing systems 36 (2023), 21702–21720

    Ma, X., Fang, G., and W ang, X.Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems 36 (2023), 21702–21720

    2023

  20. [27]

    Adell: An adaptive warp-balancing ell format for efficient sparse matrix-vector multiplication on gpus

    Maggioni, M., and Berger-Wolf, T. Adell: An adaptive warp-balancing ell format for efficient sparse matrix-vector multiplication on gpus. In 2013 42nd international conference on parallel processing (2013), IEEE, pp. 11–20

    2013

  21. [28]

    Merge-based parallel sparse matrix-vector multi- plication

    Merrill, D., and Garland, M. Merge-based parallel sparse matrix-vector multi- plication. In SC’16: Proceedings of the International Conference for High Perfor- mance Computing, Networking, Storage and Analysis (2016), IEEE, pp. 678–689

    2016

  22. [29]

    In GPU Technology Conference (2010), vol

    Naumov, M., Chien, L., V andermersch, P., and Kapasi, U.Cusparse library. In GPU Technology Conference (2010), vol. 12

    2010

  23. [30]

    In2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (2021), IEEE, pp

    Niu, Y., Lu, Z., Dong, M., Jin, Z., Liu, W., and Tan, G.Tilespmv: A tiled algorithm for sparse matrix-vector multiplication on gpus. In2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (2021), IEEE, pp. 68–78

    2021

  24. [31]

    Powerinfer: Fast large language model serving with a consumer-grade gpu

    Song, Y., Mi, Z., Xie, H., and Chen, H. Powerinfer: Fast large language model serving with a consumer-grade gpu. arXiv preprint arXiv:2312.12456 (2023)

    2023 arXiv

  25. [32]

    Sun, M., Liu, Z., Bair, A., and Kolter, J. Z. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695 (2023)

    2023 arXiv

  26. [33]

    In 2011 International conference on parallel processing (2011), IEEE, pp

    Sun, X., Zhang, Y., W ang, T., Zhang, X., Yuan, L., and Rao, L.Optimizing spmv for diagonal sparse matrices on gpu. In 2011 International conference on parallel processing (2011), IEEE, pp. 492–501

    2011

  27. [34]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  28. [35]

    Vázqez, F., Fernández, J.-J., and Garzón, E. M. A new approach for sparse matrix vector product on nvidia gpus. Concurrency and Computation: Practice and Experience 23, 8 (2011), 815–826

    2011

  29. [36]

    W., and Yelick, K

    Vuduc, R., Demmel, J. W., and Yelick, K. A. Oski: A library of automatically tuned sparse matrix kernels. In Journal of Physics: Conference Series (2005), vol. 16, IOP Publishing, p. 521

    2005

  30. [37]

    Privatelora for efficient privacy preserving llm

    Wang, Y., Lin, Y., Zeng, X., and Zhang, G. Privatelora for efficient privacy preserving llm. arXiv preprint arXiv:2311.14030 (2023)

    2023 arXiv

  31. [38]

    M.Register tiling for unstructured sparsity in neural network inference

    Wilkinson, L., Cheshmi, K., and Dehnavi, M. M.Register tiling for unstructured sparsity in neural network inference. Proceedings of the ACM on Programming Languages 7, PLDI (2023), 1995–2020

    2023

  32. [39]

    Accelerating sparse matrix computations via data compression

    Willcock, J., and Lumsdaine, A. Accelerating sparse matrix computations via data compression. In Proceedings of the 20th annual international conference on Supercomputing (2006), pp. 307–316

    2006

  33. [41]

    Sheared llama: Accelerating language model pre-training via structured pruning

    Xia, M., Gao, T., Zeng, Z., and Chen, D. Sheared llama: Accelerating language model pre-training via structured pruning. arXiv preprint arXiv:2310.06694 (2023)

    2023 arXiv

  34. [42]

    Besa: Pruning large language models with blockwise parameter- efficient sparsity allocation

    Xu, P., Shao, W., Chen, M., Tang, S., Zhang, K., Gao, P., An, F., Qiao, Y., and Luo, P. Besa: Pruning large language models with blockwise parameter- efficient sparsity allocation. In The Twelfth International Conference on Learning Representations

  35. [43]

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly

    Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., and Zhang, Y. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing (2024), 100211. Conference’17, July 2017, Washington, DC, USA Junqing Lin, Jingwei Sun, Mingge Lu,...

    2024

  36. [44]

    V., et al

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022)

    2022 arXiv

  37. [45]

    Dynamic sparse no training: Training-free fine-tuning for sparse llms

    Zhang, Y., Zhao, L., Lin, M., Sun, Y., Y ao, Y., Han, X., Tanner, J., Liu, S., and Ji, R. Dynamic sparse no training: Training-free fine-tuning for sparse llms. arXiv preprint arXiv:2310.08915 (2023)

    2023 arXiv

  38. [46]

    Acc-spmm: Accelerating general-purpose sparse matrix-matrix multiplication with gpu tensor cores

    Zhao, H., Li, S., W ang, J., Zhou, C., W ang, J., Xin, Z., Li, S., Liang, Z., Pan, Z., Liu, F., et al. Acc-spmm: Accelerating general-purpose sparse matrix-matrix multiplication with gpu tensor cores. In Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Pr...

    2025

  39. [47]

    X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al

    Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.