Pith. sign in

REVIEW 5 major objections 5 minor 19 references

Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TVM-based autotuning for the RISC-V vector extension yields schedules that beat GCC, LLVM, and muRISCV-NN on AI workloads.

desk verdict Useful integration of RVV into MetaSchedule, but the headline speedups need baseline details and variance data before they can be trusted. read the letter →

arxiv 2507.01457 v2 pith:AUJJYAG4 submitted 2025-07-02 cs.LG cs.AIcs.SE

Federico Nicolas Peccia , Frederik Haxel , Oliver Bringmann This is my paper · ORCID
classification cs.LGcs.AIcs.SE
keywords RISC-VvectorextensionRVVTVMMetaScheduletensorprogramoptimizationautotuningautovectorizationprobabilisticprograms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that probabilistic autotuning can close the gap between compiler autovectorization and hand-tuned vector libraries for RISC-V vector AI workloads. The authors integrate the RISC-V Vector Extension (RVV) into TVM's MetaSchedule framework by defining tensor intrinsics for vector-matrix multiplication and vector-vector multiply-accumulate, registering each at multiple vector lengths so the tuner can match both large and small tensors. On FPGA implementations of RISC-V SoCs with a Saturn vector unit, the schedules found by this workflow are, on average, 46% faster than GCC 14 autovectorization and 29% faster than the hand-crafted muRISCV-NN library on complete AI models, while also producing smaller binaries. On a commercial Banana Pi BPI-F3 board, the tuned programs average 35% faster than LLVM 19 autovectorization. The point is that a search-based compiler workflow can adapt to different vector-unit hardware better than fixed hand-crafted kernels.

What carries the argument

The load-bearing mechanism is the tensor intrinsic: a small named tensor operation paired with an implementation that calls RVV instructions, registered in MetaSchedule in multiple copies with halved vector-length parameters. Two intrinsics are defined: RVV_MULTIVMUL, which performs a vector-matrix multiplication by loading the input vector once, looping over rows, reducing each elementwise product, and accumulating results into the output vector register via vslideup so stores are almost eliminated; and RVV_VMACC, a vector-vector multiply-accumulate for layers without reduction. The intrinsics set LMUL = 8 and J = VLEN/32, and copies with VL = VLMAX, VLMAX/2, ..., 4 are registered so MetaSchedule's probabilistic sampling can select the appropriate vector length for any tensor shape. MetaSchedule then compiles, executes, and measures each sampled candidate on the target hardware, using the measurements to train a cost model that guides further sampling.

What would settle it

Run the same tuning workflow and the same baselines on a different RVV 1.0 board with a different microarchitecture, VLEN, or cache size, sweeping compiler optimization flags; if the tuned schedules are not consistently faster than the best autovectorized binary across models, the central claim would be overturned.

Watch

Extended reading notes

Core claim

The central claim is that RVV tensor intrinsics embedded in MetaSchedule's probabilistic program search are sufficient to beat both generic compiler autovectorization and a specialized hand-written kernel library across a range of AI workloads and vector-unit configurations. The paper demonstrates this by tuning matrix multiplications and complete networks (MLPerf Tiny tasks, MobileNetV2, ResNet18, BERT-tiny, DCGAN, MobileLLM) on FPGA-based Saturn SoCs with VLEN 256/512/1024 and on the commercial Banana Pi BPI-F3. The intrinsic for vector-matrix multiplication accumulates partial reductions in a vector register and merges them with vslideup, drastically cutting vector store instructions; the multiply-accumulate intrinsic handles non-reduction layers. By registering each intrinsic at VL = VLMAX, VLMAX/2, ..., 4, MetaSchedule can match operations of any size and choose the best vector length during tuning. The paper reports that this not only improves latency but also reduces code memory footprint by about 90% relative to muRISCV-NN for most models, and it attributes muRISCV-NN's performance degradation at larger VLEN to fixed hand-crafted kernels not adapting to hardware parameters.

Load-bearing premise

The reported speedups assume the chosen baselines and hardware platforms fairly represent the state of the art in RVV code generation, and the paper does not report repeated-run variance or compiler configuration sweeps.

Editorial extensions

If this is right

  • Autotuning can serve as a third path besides autovectorization and hand-crafted libraries for RVV AI deployment, with the advantage of re-tuning when hardware changes.
  • Because the intrinsics are datatype-generic and registered at multiple vector lengths, the same integration targets int8, float16, and float32 workloads without per-datatype hand tuning.
  • The code-size reduction makes tuned RVV schedules attractive for memory-constrained embedded devices running under microTVM/Zephyr.
  • The open-source integration gives a base for extending the same probabilistic search to other RISC-V extensions such as Packed SIMD.
  • On commercial RVV 1.0 boards, the workflow works with the full TVM runtime, so the benefit is not limited to FPGA or simulation targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The VL-halving registration trick is a general recipe: any ISA with variable vector length could be exposed to a schedule search the same way, which suggests the approach transfers to other variable-length SIMD extensions.
  • If the observed trend that muRISCV-NN degrades at larger VLEN holds for other libraries, wider vector units will increasingly favor autotuned schedules over fixed hand-written kernels, making the tuning cost a worthwhile deployment step for new hardware.
  • The main practical barrier is tuning time (9-12 seconds per candidate on the FPGA setup); a testable extension would be warm-starting the cost model from previously tuned boards to cut that cost.
  • The anomaly-detection code-size outlier shows that per-layer specialization can inflate binaries for layer-homogeneous networks; a possible improvement is sharing a single tuned kernel across identical layers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes integrating the RISC-V Vector Extension (RVV) into TVM's MetaSchedule framework by defining tensor intrinsics for vector-matrix multiplication and vector-vector multiply-accumulate, registering multiple vector-length variants, and tuning schedules on FPGA-based Saturn SoCs with VLEN 256/512/1024 and on a commercial Banana Pi BPI-F3 board. The authors report mean latency improvements of 46% over GCC 14 autovectorization and 29% over muRISCV-NN for complete models on the FPGA, and 35% over LLVM 19 autovectorization on the Banana Pi, along with code-size reductions for most tested models. They also analyze instruction traces to argue that the tuned schedules use the vector register file more efficiently than muRISCV-NN.

Significance. If the reported numbers hold up, the contribution is a useful and broad empirical demonstration that TVM/MetaSchedule can be extended to RVV targets, covering matrix multiplications, MLPerf Tiny models, CNNs, Transformers, a GAN, and a small LLM, on both FPGA-based research hardware and a commercial board. The instruction-trace analysis provides a plausible mechanistic explanation for the speedups, and the code-size reductions are a practically relevant secondary result. The main weakness is that the central claims are entirely empirical and the experiments are currently not fully auditable: compiler baseline configurations are under-specified, the stochastic tuner is run only once per workload, and no artifact link is provided. With those details supplied, this would be a solid systems contribution.

major comments (5)
  1. [Section IV, first paragraph; Figures 3, 6, 7, 10] The baseline compiler configurations are under-specified. The text states only that GCC is compiled with "-O3" and that LLVM is run with "the autovectorization feature (Non tuned (v))", but it does not report the target strings or TVM target configurations. On RISC-V, vector code generation requires explicit V-extension enablement (e.g., -march=rv64gcv) and often VLEN-aware settings (e.g., -march=rv64gcv_zvl256b or LLVM's -riscv-v-vector-bits-min) to produce RVV code. If the baselines used generic target strings, the reported mean improvements of 46%, 29%, and 35% could be inflated by partially vectorized or non-vectorized baselines. Please report the exact compiler invocations, target strings, and TVM target configuration for every baseline and for the proposed method.
  2. [Section IV-A and IV-B] MetaSchedule is a stochastic search, yet all reported latencies appear to come from a single tuning run (100 candidates for matrix multiplications, 200–400 for complete networks), with no repeated seeds or variance data. Since the headline claims are mean improvements, a single lucky or unlucky search could dominate the averages. Please run multiple tuning seeds per workload and report the mean, minimum, and variance, or otherwise demonstrate that the reported improvements are robust to the random seed.
  3. [Section III, Algorithm 1] The pseudocode in Algorithm 1 adds the C vector inside the j-loop (the vadd of C vec with out vec appears before the loop ends), so for J > 1 the original C values would be accumulated J times. In addition, the vslideup operation writes into an output register before all earlier positions have been initialized. If the actual implementation follows this pseudocode, the matrix-vector multiplication results would be incorrect for J > 1; if it does not, the pseudocode should be corrected to match the implementation. This is load-bearing because the intrinsic is the basis for all reduction-type layers in the evaluation.
  4. [Section IV-B, Figure 9 (top)] The abstract claims that the binary resulting from the proposal has a smaller code memory footprint, but Section IV-B explicitly reports that for the anomaly-detection model the proposed code is much bigger than the muRISCV-NN code because per-layer schedules are generated instead of reusing a single library function. Please qualify the code-size claim in the abstract and state the conditions under which it holds.
  5. [Throughout; abstract] The paper states "We open-sourced our proposal" but no artifact URL or repository link appears in the text or references. Given that the central claims are empirical and depend on exact baseline configurations and tuning procedures, the missing artifact link prevents independent verification of the reported numbers. Please provide a working artifact link and specify the exact version of the code used for the experiments.
minor comments (5)
  1. [Figure 7] The first subplot of Figure 7 contains the label "Runtime error" above "Latency [ms]"; this appears to be a typo for "Runtime" or a leftover debugging label and should be corrected.
  2. [Figure 9] The bottom-right panel of Figure 9 labels the x-axis "Matrices dimensions" even though the data are complete models; relabel this axis to "Model" or similar.
  3. [Footnote 1] Footnote 1 refers to "lines 15 to 22 in Figure 1", but the pseudocode with those lines is Algorithm 1; the cross-reference should be corrected.
  4. [Section IV, first paragraph; Figures 3 and 7] The "Non tuned" baseline used for the speedup-percentage plots is compiled with -Os, while the textual claims compare against -O3 autovectorization. Please clarify which baseline is used in the plotted improvement percentages, or use a speed-optimized non-vector baseline so the percentages are not conflated with a size-optimized baseline.
  5. [Section IV-A and IV-B] Minor language issues: "In average" should be "On average" in Section IV-A, and "probable more efficient solutions" should be "probably more efficient solutions" in Section IV-B.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported speedups are empirical measurements against external baselines and are not derived from the paper's own inputs.

full rationale

This is a build-and-measure systems paper. The central claims are that schedules found by extending TVM's MetaSchedule with custom RVV tensor intrinsics achieve lower latency than GCC 14 autovectorization, muRISCV-NN, and LLVM 19 autovectorization on specific FPGA and commercial hardware. These claims are supported by direct measurements of generated code on real or FPGA-implemented targets, not by a derivation that reduces to the paper's inputs. The design constants (LMUL = 8, the VL halving chain down to 4, and J = VLEN/32) are presented as engineering choices or empirically observed thresholds, and they are not fitted to the reported speedup numbers. No equation in the paper defines the measured improvements in terms of the intrinsics' parameters, and no load-bearing argument relies on a self-citation or an imported uniqueness theorem. The paper's references to TVM, MetaSchedule, muRISCV-NN, Chipyard, Saturn, and the benchmark suites are external prior work that provides tools and baselines, not the conclusions of this paper. Concerns about under-specified compiler flags or single tuning runs are auditability and reproducibility issues, not circularity. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its free parameters are engineering constants chosen for the RVV integration, not data-fit coefficients. The axiomatic assumptions are standard for empirical compiler studies; the most fragile are the baseline representativeness and the convergence of the stochastic search.

free parameters (4)
  • LMUL = 8 (vector register group multiplier) = 8
    Chosen by the authors as the maximum LMUL for all tensor intrinsics (Section III). It bounds the number of elements per instruction and is a design choice, not derived from a theorem.
  • Minimum VL for intrinsic registration = 4 elements
    Intrinsics are registered with VL halved from VLMAX down to 4; the cutoff is justified only as 'we found empirically' (Section III).
  • J for matrix-vector multiplication intrinsic = VLEN/32, plus J=1 variant
    J controls the output tile width and is set to VLEN/32 to fill a vector register, with a J=1 fallback for small matrices (Section III).
  • MetaSchedule tuning budget = 100 iterations (matmul), 200 (networks), 400 (LLM)
    The number of candidate schedules evaluated per workload sets the search effort and therefore the achieved speedup; results are reported at these budgets without sensitivity analysis (Sections IV-A, IV-B).
assumptions (5)
  • domain assumption The RVV ISA semantics for vle, vmul, vredsum, vslideup, vmacc, and the VLEN/SEW/LMUL relationship behave as documented on the tested hardware
    Section II and Algorithm 1 rely on these semantics; any microarchitectural divergence could change measured latencies.
  • domain assumption MetaSchedule's probabilistic search and cost model converge to near-optimal schedules within the allotted candidate budgets
    The paper reports the best candidate from 100 to 400 samples but does not characterize the search distribution or show convergence (Sections II, IV).
  • domain assumption GCC 14 and LLVM 19 autovectorization, used at default or unspecified settings, are representative strong baselines for RVV code generation
    The head-to-head improvement percentages are computed against these specific toolchain versions and settings (Section IV).
  • domain assumption Instruction traces from the QEMU TCG plugin faithfully represent the executed instruction mix of the binaries
    Used in the analysis of register-file efficiency and store reduction (Section IV-A, Figures 5 and 9).
  • domain assumption The Zephyr/microTVM runtime and TVM runtime do not introduce overheads that systematically favor one implementation over another
    Latency comparisons are end-to-end executions on each platform but no profiling of overheads is reported (Section IV).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs." pith.science (2026). https://pith.science/paper/AUJJYAG4

@misc{pith2026250701457,
  author       = {Pith},
  title        = {Pith review of: Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AUJJYAG4}},
  note         = {Machine review of arXiv:2507.01457}
}
read the original abstract

RISC-V provides a flexible and scalable platform for applications ranging from embedded devices to high-performance computing clusters. Particularly, its RISC-V Vector Extension (RVV) becomes of interest for the acceleration of AI workloads. But writing software that efficiently utilizes the vector units of RISC-V CPUs without expert knowledge requires the programmer to rely on the autovectorization features of compilers or hand-crafted libraries like muRISCV-NN. Smarter approaches, like autotuning frameworks, have been missing the integration with the RISC-V RVV extension, thus heavily limiting the efficient deployment of complex AI workloads. In this paper, we present a workflow based on the TVM compiler to efficiently map AI workloads onto RISC-V vector units. Instead of relying on hand-crafted libraries, we integrated the RVV extension into TVM's MetaSchedule framework, a probabilistic program framework for tensor operation tuning. We implemented different RISC-V SoCs on an FPGA and tuned a wide range of AI workloads on them. We found that our proposal shows a mean improvement of 46% in execution latency when compared against the autovectorization feature of GCC, and 29% against muRISCV-NN. Moreover, the binary resulting from our proposal has a smaller code memory footprint, making it more suitable for embedded devices. Finally, we also evaluated our solution on a commercially available RISC-V SoC implementing the RVV 1.0 Vector Extension and found our solution is able to find mappings that are 35% faster on average than the ones proposed by LLVM. We open-sourced our proposal for the community to expand it to target other RISC-V extensions.

Figures

Figures reproduced from arXiv: 2507.01457 by the authors.

Figure 1
Figure 1. For the standard vector-vector elementwise multipli [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Relationship between hardware (VLEN) and soft [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Benchmarking of matrix multiplications on the Saturn Vector Unit (VLEN=1024). Speedup is calculated using ”Non [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Impact of VLEN on the execution time of matrix [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Analysis of the instruction execution traces measured [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Benchmarking of matrix multiplications on the Banana Pi-F3 (VLEN=256) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Complete models on the Saturn Vector Unit (VLEN=1024). Improvement is calculated using ”Non tuned” as baseline. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Analysis of the instruction execution traces measured [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Complete models on the Banana Pi-F3 (VLEN=256). [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 10 canonical work pages

  1. [1]

    Xuantie-910: A commercial multi-core 12-stage pipeline out-of-order 64-bit high performance risc-v processor with vector extension : Indus- trial product,

    C. Chen, X. Xiang, C. Liu, Y . Shang, R. Guo, D. Liu, Y . Lu, Z. Hao, J. Luo, Z. Chen, C. Li, Y . Pu, J. Meng, X. Yan, Y . Xie, and X. Qi, “Xuantie-910: A commercial multi-core 12-stage pipeline out-of-order 64-bit high performance risc-v processor with vector extension : Indus- trial product,” in 2020 ACM/IEEE 47th Annual International Symposium on Compu...

    work page 2020

  2. [2]

    banana pi bpi-f3 [Online]

    Guangdong bipai technology co. banana pi bpi-f3 [Online]. Available: https://docs.banana-pi.org/en/BPI-F3/BananaPi BPI-F3. [Accessed: 16 April 2025]

    work page 2025

  3. [3]

    Instruction scheduling in the saturn vector unit,

    J. Zhao, D. Grubb, M. Rusch, T. Wei, K. Anderson, B. Nikolic, and K. Asanovic, “Instruction scheduling in the saturn vector unit,” 2024. [Online]. Available: https://arxiv.org/abs/2412.00997

    arXiv 2024

  4. [4]

    Spatz: Clustering compact risc-v-based vector units to maximize computing efficiency,

    M. Cavalcante, M. Perotti, S. Riedel, and L. Benini, “Spatz: Clustering compact risc-v-based vector units to maximize computing efficiency,” Sep. 2023

    work page 2023

  5. [5]

    A “new ara

    M. Perotti, M. Cavalcante, N. Wistoff, R. Andri, L. Cavigelli, and L. Benini, “A “new ara” for vector computing: An open source highly efficient risc-v v 1.0 vector processor design,” in 2022 IEEE 33rd International Conference on Application-specific Systems, Architectures and Processors (ASAP) , 2022, pp. 43–51

    work page 2022

  6. [6]

    Performance left on the table: An evaluation of compiler autovectorization for risc-v,

    N. Adit and A. Sampson, “Performance left on the table: An evaluation of compiler autovectorization for risc-v,” IEEE Micro , vol. 42, no. 5, pp. 41–48, 2022

    work page 2022

  7. [7]

    muriscv-nn: Challenging zve32x autovectorization with tinyml inference library for risc-v vector extension,

    P. van Kempen, J. P. Jones, D. Mueller-Gritschneder, and U. Schlichtmann, “muriscv-nn: Challenging zve32x autovectorization with tinyml inference library for risc-v vector extension,” in Proceedings of the 21st ACM International Conference on Computing Frontiers: Workshops and Special Sessions , ser. CF ’24 Companion. New York, NY , USA: Association for C...

    work page arXiv 2024

  8. [8]

    Learning to Optimize Tensor Programs

    T. Chen, L. Zheng, E. Q. Yan, Z. Jiang, T. Moreau, L. Ceze, C. Guestrin, and A. Krishnamurthy, “Learning to optimize tensor programs,” CoRR, vol. abs/1805.08166, 2018. [Online]. Available: http://arxiv.org/abs/1805.08166

    work page Pith review arXiv 2018

Show all 19 references
  1. [9]

    Tensor program optimization with probabilistic programs,

    J. Shao, X. Zhou, S. Feng, B. Hou, R. Lai, H. Jin, W. Lin, M. Masuda, C. H. Yu, and T. Chen, “Tensor program optimization with probabilistic programs,” 2022. [Online]. Available: https://arxiv.org/abs/2205.13603

    2022 arXiv

  2. [10]

    Chipyard: Integrated design, simulation, and implementation framework for custom socs,

    A. Amid, D. Biancolin, A. Gonzalez, D. Grubb, S. Karandikar, H. Liew, A. Magyar, H. Mao, A. Ou, N. Pemberton, P. Rigge, C. Schmidt, J. Wright, J. Zhao, Y . S. Shao, K. Asanovi´c, and B. Nikoli ´c, “Chipyard: Integrated design, simulation, and implementation framework for custo...

    2020

  3. [11]

    zephyr [Online]

    Linux foundation. zephyr [Online]. Available: https://www. zephyrproject.org/. [Accessed: 16 April 2025]

    2025

  4. [13]

    Mlperf tiny benchmark,

    C. R. Banbury, V . J. Reddi, P. Torelli, J. Holleman, N. Jeffries, C. Kir ´aly, P. Montino, D. Kanter, S. Ahmed, D. Pau, U. Thakker, A. Torrini, P. Warden, J. Cordaro, G. D. Guglielmo, J. M. Duarte, S. Gibellini, V . Parekh, H. Tran, N. Tran, W. Niu, and X. Xu, “Mlperf tiny be...

    2021 arXiv

  5. [15]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015. [Online]. Available: https://arxiv.org/abs/1512.03385

    2015 arXiv

  6. [16]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019. [Online]. Available: https://arxiv.org/abs/1810.04805

    2019 arXiv

  7. [17]

    Unsupervised representation learning with deep convolutional generative adversarial networks,

    A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,”

  8. [18]

    Mobilellm: Optimizing sub-billion parameter language models for on-device use cases,

    Z. Liu, C. Zhao, F. Iandola, C. Lai, Y . Tian, I. Fedorov, Y . Xiong, E. Chang, Y . Shi, R. Krishnamoorthi, L. Lai, and V . Chandra, “Mobilellm: Optimizing sub-billion parameter language models for on-device use cases,” 2024. [Online]. Available: https: //arxiv.org/abs/2402.14905

    2024 arXiv

  9. [2016]

    Available: https://arxiv.org/abs/1511.06434

    [Online]. Available: https://arxiv.org/abs/1511.06434

    arXiv

  10. [2017]

    Available: http://arxiv.org/abs/1712.05877

    [Online]. Available: http://arxiv.org/abs/1712.05877

    arXiv

  11. [2018]

    Available: http://arxiv.org/abs/1801.04381

    [Online]. Available: http://arxiv.org/abs/1801.04381

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.