Pith. sign in

REVIEW 2 major objections 5 minor 37 references

Distributed and heterogeneous tensor-vector contraction algorithms for high performance computing

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A distributed tensor-vector contraction that stays oblivious to contraction mode, split direction, and tensor order, sustaining 50-80% of peak memory bandwidth — plus a three-buffer HOPM that cuts streamed memory by an order of magnitude.

desk verdict Solid dHOPM3 contribution, but the headline dTVC bandwidths skip the k=s collective, overstating the 'oblivious' claim. read the letter →

arxiv 2501.03121 v2 pith:WNEP2GZQ submitted 2025-01-06 cs.DC

Pedro J. Martinez-Ferrer , Albert-Jan Yzelman , Vicenç Beltran This is my paper · ORCID
classification cs.DC
keywords tensorcontractiondistributedmemoryhighbandwidthmixedprecisionGPUtask-basedparallelizationhigher-orderpowermethod
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the most memory-hungry operation in tensor computing — contracting a dense multidimensional array against a vector along one of its axes — can run on distributed CPU clusters at the speed the memory hardware actually delivers, and that this stays true regardless of which axis is contracted, which axis splits the data across processes, or how many axes the tensor has. It further claims that the higher-order power method, the standard algorithm for the best rank-one tensor approximation, can be reorganized around three buffers so that it streams up to an order of magnitude less memory than a canonical distributed implementation while moving only about twice the data of a single tensor-vector contraction. If these claims are right, distributed tensor decompositions used in data analysis stop paying the usual penalty of reshuffling data across processes and become purely memory-bound, which is the best possible regime for this class of kernel. The support consists of analytical streamed-memory formulas plus measurements on three architectures that place the new routines at 50-80% of theoretical peak bandwidth, on par with standard memory-bandwidth benchmark figures.

What carries the argument

Three objects carry the argument. First, the native TVC kernel: instead of calling batched BLAS matrix-vector routines, it reinterprets the tensor $\mathcal{A}$ through its matricized form $\mathbf{A}_{uv \times n_k}$, with $u = \prod_{i<k} n_i$ and $v = \prod_{i>k} n_i$, and fans the resulting independent vector-matrix or matrix-vector products across cores so that every contraction mode receives the same treatment. Second, one-dimensional splitting: the tensor is cut along a single dimension $s$ for MPI distribution, producing two regimes — the benign case $k \neq s$, where each process emits a disjoint slice of the output, and the costly case $k = s$, where full-size partial outputs must be summed globally. Third, the three-buffer scheme of dHOPM3 (Algorithm 1): a buffer $W^{(p)}$ carries previously computed intermediate tensors into the next contraction, so that $(d-1)(d-2)/2$ of the largest contractions of the canonical HOPM are skipped, while analytical streamed-memory formulas (Eqs. 3-7) quantify how touched memory scales with process count $p$, splitting dimension $s$, and tensor order $d$. Task-based parallelization with explicit data dependencies is what lets consecutive TVC operations overlap across threads instead of synchronizing at every contraction.

What would settle it

Run dTVC or dHOPM3 on a deliberately non-hypersquare tensor whose smallest mode is tiny (for instance, size two in one axis and on the order of $10^9$ elements in the other modes) and try to scale past two MPI processes: if the extra processes stay idle or the per-process bandwidth collapses, the claimed obliviousness to splitting and the 50-80% bandwidth figures are confined to tensors with large, balanced modes.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that a native, non-BLAS tensor-vector contraction (TVC) kernel that distributes the column space of the tensor's matricized view across cores can be lifted to distributed memory through one-dimensional MPI splitting while remaining oblivious to the contraction mode $k$, the splitting dimension $s$, and the tensor order $d$: averaged over all contraction modes and all splitting dimensions for dense hypersquare tensors of orders 2 through 10, the distributed kernel dTVC sustains roughly 70-85% of theoretical peak memory bandwidth on a general-purpose CPU cluster and 50-70% on an HBM-equipped ARM cluster, which the paper reports as on par with standard triad bandwidth figures for those machines. The companion algorithm dHOPM3, a three-buffer distributed higher-order power method, exploits the distributive property of consecutive contractions so that partial results stay distributed and a global output tensor is never assembled inside the iteration, reuses previously computed intermediate tensors through a third buffer, and saves up to one order of magnitude of streamed memory relative to a canonical distributed HOPM while running at roughly half the throughput of a single dTVC. The paper also claims this is the first time a distributed TVC has been shown to remain oblivious to MPI splitting, contraction mode, and tensor order at once, provided the tensor sizes do not vary significantly.

Load-bearing premise

The design assumes one-dimensional tensor splitting is the right trade, which caps the usable process count at the size of the largest tensor mode — a real tensor with a small mode (say, size two along one axis) can engage at most two processes, so the near-peak bandwidth and strong-scalability figures measured up to 128 processes do not transfer to such tensors, and the analytical memory model further assumes balanced, hypersquare-like mode sizes that the paper only calls 'representative'.

Editorial extensions

If this is right

  • A single distributed kernel covers all modes and splits: on CPUs, users of dTVC get essentially the same bandwidth whether they contract along the first, middle, or last axis and distribute along any one axis, so mode-aware tuning of the kernel becomes unnecessary.
  • Split against contiguity: the analytical formulas show that splitting along the last dimension minimizes streamed memory for the distributed HOPM and approaches $M_{\text{seq}}/p$, while splitting along the first dimension roughly doubles data movement at high process counts — the opposite of what a naive contiguity-preserving choice would pick.
  • The cost of a full distributed HOPM becomes predictable: dHOPM3's throughput sits about a factor of two below a single dTVC for every tensor order tested, because the runtime is dominated by its first two contractions, so the whole algorithm's cost can be forecast from the TVC kernel's bandwidth.
  • Mixed precision pays even without hardware support: storing tensors low and computing high, with conversions staged through an aligned cache buffer, doubles throughput with single precision and roughly quadruples it with brain-float storage on CPUs, with half-float slightly behind.
  • GPUs are not automatically faster: because looped batched CUDA kernels suffer launch overhead and dHOPM3 contains synchronous collectives, the native multi-core CPU implementations match or exceed the GPU versions on strong scaling, and general-purpose DDR4 systems compete with HBM-equipped clusters on these workloads.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The real ceiling is the smallest mode: because the design keeps one-dimensional splitting, the maximum useful process count equals the largest mode size, so a tensor with a mode of size two can never engage more than two processes, and the strong-scalability curves up to 128 processes apply only to tensors whose every mode is large enough to split.
  • The 'representative' clause carries weight: the memory model is derived and visualized for hypersquare tensors, and the paper concedes it stays representative only when mode sizes are of the same order of magnitude, so severely elongated tensors could overturn the recommended choice of splitting dimension in practice.
  • The three-buffer idea is exportable: any algorithm that chains contractions over shared intermediate results — for instance the inner iterations of alternating least squares or of HOOI — could use the same windowed-buffer trick, since the saving comes from not re-streaming intermediates rather than from anything specific to HOPM.
  • The staging-buffer pattern for mixed precision is a portable recipe: vectorize the high-precision compute, hold results in an aligned array, then convert back in a separate unrolled loop, which is exactly the structure that lets low-precision storage pay off on CPUs without native hardware support.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript extends the authors' prior shared-memory native TVC algorithm to distributed-memory systems. The proposed dTVC splits a dense tensor along a single dimension, contracts against a vector, and distinguishes the cases where the contraction mode coincides with the splitting dimension (k=s) from the cases where it does not; the k=s case requires a global reduction. The paper also proposes dHOPM3, a three-buffer distributed higher-order power method that keeps intermediate tensors distributed and delays the Eq. (2) summation until the normalization step. Alongside the algorithms, the authors derive analytical streamed-memory formulas for 1D splitting, benchmark dTVC and dHOPM3 on up to 128 MPI processes on MN4 (Intel), CTE-ARM (A64FX HBM), and CTE-POWER (Volta GPUs), compare against CTF, MKL, and cuBLAS, and evaluate mixed-precision variants.

Significance. If the claims hold, the contribution is useful: it provides an open-source distributed TVC/HOPM library, a mode-oblivious distributed TVC kernel, analytical formulas for the memory effects of 1D splitting, and strong-scalability experiments cross-checked against STREAM, MKL, cuBLAS, and CTF. The CPU bandwidth figures (about 50%–85% of peak, with low variance across modes and orders) are credible and well aligned with STREAM measurements, and the comparison against external libraries is a strength. The paper also has clear limitations that it honestly acknowledges: 1D splitting caps the number of processes at max(n_i), the analytical memory model assumes representative hypersquare tensors, and the main dTVC benchmarks deliberately omit the global construction/reduction of the output tensor. The central quantitative claims nevertheless overreach the measurements in one load-bearing respect, as detailed below.

major comments (2)
  1. [§5.2–§5.3, Eq. (2), Table 2, Fig. 5] The dTVC benchmarks are presented as evidence that dTVC sustains 50–85% of peak bandwidth and is 'oblivious to MPI splitting, contraction mode, and tensor order.' However, the measurements intentionally ignore the global construction of the final tensor, and §5.2 explicitly states that the global reduction of Eq. (2) was not optimized. Since §4.1 defines k=s as a sum over p partial subtensors requiring a collective summation, roughly one in d of all mode/splitting combinations averaged into Table 2 and Fig. 5 is a k=s case measured without the reduction. Consequently, the 'zero MPI communication parts' characterization in §5.3 is not compatible with the paper's own definition of a complete dTVC for k=s. The claim as written is established only for the local contraction kernel in the k≠s cases and for an incomplete operation in the k=s cases. Please either re-benchmark the full dTVC including the reduction and report the k=s cases separately, or re-scope the abstract and Section 5.1 claims so that they refer to the local kernel with the k=s restriction stated explicitly.
  2. [§4.2, Eqs. (3)–(7), Fig. 2] The abstract and §4.2 state that dHOPM3 'can save up to one order of magnitude of streamed memory' over a canonical distributed HOPM. This factor is derived entirely from the analytical model: Fig. 2 plots ratios of touched memory predicted by Eqs. (3)–(7), and the subsequent performance experiments measure time and throughput rather than actual streamed bytes. Since this memory-saving figure is a headline claim of the paper, it needs either empirical support, for example by instrumenting the kernels or using hardware performance counters to measure memory traffic on at least a few of the Table 1 tensors, or an equally explicit qualification in the abstract and conclusions that the factor is an analytical prediction for hypersquare tensors with balanced mode sizes and has not been directly measured. Without one of these changes, the abstract presents a model-derived quantity in the same register as the measured bandwidth results.
minor comments (5)
  1. [§5.3] The word 'matriziced' in the sentence about the matricized tensor's shape should be 'matricized.'
  2. [§5.5] The word 'demostrating' in the discussion of the half-single precision results should be 'demonstrating.'
  3. [§5.2] The sentence 'we have not optimized neither the disjoint union, Eq.(1), nor the global reduction, Eq.(2)' contains a double negative; rewrite as 'we have optimized neither the disjoint union, Eq. (1), nor the global reduction, Eq. (2).'
  4. [§4.1] The strong-scalability conclusions should be accompanied by an explicit reminder that, because 1D splitting limits p to max(n_i), a tensor with a small mode (for example, n_k = 2) can use at most two processes; the current text acknowledges the disadvantage but does not state its direct consequence for the generality of the reported scaling curves.
  5. [§4.2, Eqs. (6)–(7)] The index ranges in Eqs. (6) and (7) are hard to verify from the surrounding text; a brief derivation or a small worked example for d=3 and d=4 would improve reproducibility of the analytical results.

Circularity Check

1 steps flagged · score 6.0 of 10

Partially circular: dTVC's headline obliviousness and bandwidth result is obtained from a benchmark that omits the k=s collective its own Eq. (2) requires; dHOPM3 memory analysis remains self-contained.

  1. self definitional [Section 4.1 (Eq. 2) versus Section 5.3 (Table 2 and Fig. 5)]
    "Additionally, Eq.(2) demands a collective summation of all these equally-sized subtensors, element by element, to get the global tensor ... Table 2 reports on the results of the dTVC ignoring the global construction of the final tensor ... It is worth highlighting that dTVC remains an embarrassingly parallel application (the final disjoint union operation is discarded) with zero MPI communication parts."

    Eq. (2) defines the k=s case of dTVC as a sum over p partial subtensors followed by 'a collective summation' to obtain the global tensor. Section 5.3 benchmarks dTVC 'ignoring the global construction of the final tensor' and then calls dTVC 'embarrassingly parallel ... with zero MPI communication parts.' Figures 5 and Table 2 average over all contraction modes and all splitting dimensions, so about 1/d of the averaged cases are k=s and are measured without the collective the definition requires. The headline claim that dTVC is 'oblivious to MPI splitting' and the 50–85% of peak bandwidth figures are therefore true by construction of the benchmark, not for the complete distributed TVC defined in Eq. (2).

full rationale

The main circularity is localized to the dTVC performance and obliviousness claim. Eq. (2) states that k=s dTVC requires a collective summation, yet Section 5.3 reports bandwidth while 'ignoring the global construction of the final tensor' and describes dTVC as embarrassingly parallel with zero MPI communication parts because 'the final disjoint union operation is discarded.' Averaging over all contraction modes and splitting dimensions then makes 'oblivious to MPI splitting' hold by construction for a local kernel rather than for a complete distributed operation. I score this as 6 rather than higher because the dHOPM3 streamed-memory analysis (Eqs. 3–7 and Fig. 2) is a self-contained derivation from the algorithm's buffer and contraction counts; the mixed-precision and dHOPM3 throughput results are empirical and checked against external STREAM and cuBLAS references; and the self-citations [7,8] are published prior work with code and are not used as an unverified uniqueness argument. The k=s omission is also a correctness caveat: the 'first time a distributed TVC remains oblivious to MPI splitting' assertion is not established for complete distributed TVCs, though the underlying native kernel and dHOPM3 algorithm are not derived from their own conclusions.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claims rest on standard assumptions about the memory-bound nature of TVC, a dense row-major tensor layout, and the sufficiency of 1D MPI splitting. No fitted physical constants or invented entities appear. The main hand-tuned elements are the vector-alignment splitting heuristic and task block sizes, which are disclosed but not systematically swept.

free parameters (3)
  • Vector-length-aligned splitting quotient = p reduced, e.g., [4/3] -> 4/2 (Section 4.1)
    The 'optimal division' heuristic uses ceiling division then lowers p so chunk sizes are multiples of the 512-bit SIMD vector length (8 doubles). This hand-chosen alignment affects the actual number of MPI processes used in strong-scaling runs (e.g., order 7-9 tensors use fewer processes), so it shapes the reported scalability curves.
  • TVC task block sizes (bsM, bsN) = Not reported numerically
    Task granularity in Section 4.3 is chosen such that there are more tasks than threads; the exact block sizes are not given, so the overlap behavior is not reproducible without inspecting the source.
  • Mixed-precision cache width unrollY = simdObjSz^2 (Section 5.5 code snippet)
    Implementation constant for the aligned intermediate buffer in axpby; not a scientific parameter but part of the performance-critical design.
assumptions (6)
  • domain assumption TVC is memory-bound with arithmetic intensity 1-2 FLOP/byte.
    Section 1: this is why performance is measured in GB/s and compared to STREAM.
  • domain assumption The input tensor is dense and stored in last-order, nonhierarchical layout (C-style row-major equivalent).
    Section 3: the entire dTVC algorithm and the splitting analysis depend on this layout; other layouts (Morton order from ref [6]) are outside scope.
  • domain assumption One-dimensional splitting yields minimum communication and best computational performance.
    Section 4.1: cited to the authors' own ref [7]. This justifies restricting to 1D cuts; it is load-bearing for the scalability and the max(ni) process limit.
  • domain assumption The input vector x can be harmlessly duplicated on all processes because uv >> nk.
    Section 4.1: if the contracted mode is large relative to the product of the other modes, duplicating x is not harmless and the memory model (Eq. 4) would need revision.
  • domain assumption The streamed-memory model assumes hypersquare tensors; for non-hypersquare tensors the results are representative only if all mode sizes are of the same order of magnitude.
    Section 4.2, after Eq. (7). The analytical claims (Figs. 2, H-1 ratios) and the experimental tensors (Table 1) all satisfy this assumption.
  • domain assumption MPI collective costs are modeled with the ring algorithm.
    Section 4.2: the 4n(p-1)/p extra touched memory per process for allreduce/allgather assumes bandwidth-optimal ring collectives, which may not match the vendor MPI libraries used in experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Distributed and heterogeneous tensor-vector contraction algorithms for high performance computing." pith.science (2026). https://pith.science/paper/WNEP2GZQ

@misc{pith2026250103121,
  author       = {Pith},
  title        = {Pith review of: Distributed and heterogeneous tensor-vector contraction algorithms for high performance computing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNEP2GZQ}},
  note         = {Machine review of arXiv:2501.03121}
}
read the original abstract

The tensor-vector contraction (TVC) is the most memory-bound operation of its class and a core component of the higher-order power method (HOPM). This paper brings distributed-memory parallelization to a native TVC algorithm for dense tensors that overall remains oblivious to contraction mode, tensor splitting and tensor order. Similarly, we propose a novel distributed HOPM, namely dHOPM3, that can save up to one order of magnitude of streamed memory and is about twice as costly in terms of data movement as a distributed TVC operation (dTVC) when using task-based parallelization. The numerical experiments carried out in this work on three different architectures featuring multi-core and accelerators confirm that the performances of dTVC and dHOPM3 remain relatively close to the peak system memory bandwidth (50%-80%, depending on the architecture) and on par with STREAM benchmark figures. On strong scalability scenarios, our native multi-core implementations of these two algorithms can achieve similar and sometimes even greater performance figures than those based upon state-of-the-art CUDA batched kernels. Finally, we demonstrate that both computation and communication can benefit from mixed precision arithmetic also in cases where the hardware does not support low precision data types natively.

Figures

Figures reproduced from arXiv: 2501.03121 by the authors.

Figure 1
Figure 1. One-dimensional optimal splittings (s = 0, 1, 2) using up to three processes (p = 3) for a third-order tensor A2×3×4 stored in last-order, nonhierarchical memory layout. 4.1. Distributed-memory tensor–vector contraction To allow for distributed-memory TVC computations, one can split the given input tensor A along one or more dimensions. Such splitting also influences the output ten￾sor Y that is distributed in a sim… view at source ↗
Figure 2
Figure 2. (a) memory ratio (η−1 ) incurred by the classical, dHOPM implementation and (b) memory ratio (H−1 ) between this canonical version and dHOPM3 as a function of the nondimensional number of processes (pˆ) and the nondimensional splitting dimension (sˆ). Results correspond to the third- and tenth-order tensors of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Paraver traces from the hybrid execution of (a) the dHOPM [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: dTVC normalized throughput (including global assembly of the output tensor) measured as the number of iterations per second per [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: dTVC normalized, averaged bandwidth and averaged throughput over all contraction modes and all splitting dimensions for the [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: dHOPM3 normalized bandwidth and throughput for the tensors of [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: dHOPM3 normalized throughput for the tensors of [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 23 canonical work pages

  1. [1]

    T. G. Kolda, B. W. Bader, Tensor decompositions and applica- tions, SIAM Review 51 (2009) 455–500. doi:10.1137/07070111X

    doi:10.1137/07070111x 2009

  2. [2]

    C. G. Khatri, C. R. Rao, Solutions to some functional equations and their applications to characterization of probability distri- butions, Sankhy¯ a: The Indian Journal of Statistics, Series A (1961-2002) 30 (1968) 167–180

    work page 1968

  3. [3]

    L. D. Lathauwer, B. D. Moor, J. Vandewalle, On the best rank-1 and rank-(R1,R2,...,R N) approximation of higher-order tensors, SIAM J. Matrix Anal. Appl. 21 (2000) 1324–1342. doi:10.1137/S0895479898346995

    doi:10.1137/s0895479898346995 2000

  4. [4]

    L. R. Tucker, Some mathematical notes on three-mode fac- tor analysis, Psychometrika 31 (1966) 279–311. doi:10.1007/ bf02289464

    work page 1966

  5. [5]

    explanatory

    R. A. Harshman, Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-model factor analysis, in: UCLA Working Papers in Phonetics, volume 16, 1970, pp. 1–84

    work page 1970

  6. [6]

    Pawłowski, B

    F. Pawłowski, B. Uçar, A. N. Yzelman, A multi-dimensional Morton-ordered block storage for mode-oblivious tensor com- putations, Journal of Computational Science 33 (2019) 34–44. doi:10.1016/j.jocs.2019.02.007

    work page doi:10.1016/j.jocs.2019.02.007 2019

  7. [7]

    Pawłowski, B

    F. Pawłowski, B. Uçar, A. N. Yzelman, High performance tensor–vector multiplication on shared-memory systems, in: Parallel Processing and Applied Mathematics, 2020, pp. 38–48. doi:10.1007/978-3-030-43229-4_4

    work page doi:10.1007/978-3-030-43229-4_4 2020

  8. [8]

    P. J. Martinez-Ferrer, A.-J. Nicholas Yzelman, V. Beltran, A na- tive tensor-vector multiplication algorithm for high performance computing, IEEE Transactions on Parallel and Distributed Systems 33 (2022) 3363–3374. doi:10.1109/TPDS.2022.3153113

    arXiv 2022

Show all 37 references
  1. [9]

    K. Shin, B. Hooi, J. Kim, C. Faloutsos, Detecting group anoma- lies in tera-scale multi-aspect data via dense-subtensor mining, Frontiers in Big Data 3 (2021). doi:10.3389/fdata.2020.594302

    2021

  2. [10]

    P. J. Martinez-Ferrer, dTVC library (version 1.0) [Computer software], Code Ocean, 2024. doi:10.24433/CO.8844920.v1

    2024 doi

  3. [11]

    Di Napoli, D

    E. Di Napoli, D. Fabregat-Traver, G. Quintana-Ortí, P. Bienti- nesi, Towards an efficient use of the BLAS library for multilinear tensor contractions, Applied Mathematics and Computation 235 (2014) 454–468. doi:https://doi.org/10.1016/j.amc.2014.02. 051

    2014 doi

  4. [12]

    Bassoy, Design of a high-performance tensor-vector multi- plication with BLAS, in: Computational Science – ICCS 2019, 2019, pp

    C. Bassoy, Design of a high-performance tensor-vector multi- plication with BLAS, in: Computational Science – ICCS 2019, 2019, pp. 32–45. doi:10.1007/978-3-030-22734-0_3

    2019 doi

  5. [13]

    Xianyi, W

    Z. Xianyi, W. Qian, Z. Yunquan, Model-driven level 3 BLAS performance optimization on Loongson 3A processor, in: Proceedings of the 2012 IEEE 18th International Confer- ence on Parallel and Distributed Systems, 2012, pp. 684–691. doi:10.1109/ICPADS.2012.97

    2012 doi

  6. [14]

    F. G. Van Zee, R. A. van de Geijn, BLIS: A framework for rapidly instantiating BLAS functionality, ACM Transactions on Mathematical Software 41 (2015) 14:1–14:33. 15

    2015

  7. [15]

    Heinecke, G

    A. Heinecke, G. Henry, M. Hutchinson, H. Pabst, LIBXSMM: Accelerating small matrix multiplications by runtime code gener- ation, in: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis,

  8. [16]

    U. Kang, E. Papalexakis, A. Harpale, C. Faloutsos, Gigatensor: Scaling tensor analysis up by 100 times - algorithms and discov- eries, in: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012, pp. 316–324. doi:10.1145/2339530.2339583

    2012

  9. [17]

    N. Park, B. Jeon, J. Lee, U. Kang, BIGtensor: Mining billion- scale tensor made easy, in: Proceedings of the 25th ACM International on Conference on Information and Knowledge Man- agement, 2016, pp. 2457–2460. doi:10.1145/2983323.2983332

    2016

  10. [18]

    Blanco, B

    Z. Blanco, B. Liu, M. M. Dehnavi, CSTF: Large-scale sparse tensor factorizations on distributed platforms, in: Proceedings of the 47th International Conference on Parallel Processing, 2018. doi:10.1145/3225058.3225133

    2018

  11. [19]

    doi:10.1016/0167-8191(94)90033-7

    D.W.Walker, Thedesignofastandardmessagepassinginterface for distributed memory concurrent computers, Parallel Comput- ing 20 (1994) 657–673. doi:10.1016/0167-8191(94)90033-7

    1994 doi

  12. [20]

    A. N. Yzelman, D. Di Nardo, J. M. Nash, W. J. Suijlen, A C++ GraphBLAS: specification, implementation, parallelisation, and evaluation, 2020. arXiv:1906.03196

    2020 arXiv

  13. [21]

    Brock, A

    B. Brock, A. Buluç, T. Mattson, S. McMillan, J. Moreira, The GraphBLAS C API specification (Version 2.0.0) [Computer soft- ware], 2021. URL: https://graphblas.org/docs/GraphBLAS_ API_C_v2.0.0.pdf

    2021

  14. [22]

    D. G. Spampinato, D. Jelovina, J. Zhuang, A.-J. N. Yzelman, Towards structured algebraic programming, in: Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming, 2023, pp. 50–61. doi:10.1145/3589246.3595373

    2023

  15. [23]

    Solomonik, D

    E. Solomonik, D. Matthews, J. R. Hammond, J. F. Stanton, J. Demmel, A massively parallel tensor contraction framework for coupled-cluster computations, Journal of Parallel and Dis- tributed Computing 74 (2014) 3176–3190. doi:10.1016/j.jpdc. 2014.06.002

    2014 doi

  16. [24]

    J. A. Calvin, C. A. Lewis, E. F. Valeev, Scalable task-based algo- rithm for multiplication of block-rank-sparse matrices, in: Pro- ceedings of the 5th Workshop on Irregular Applications: Archi- tectures and Algorithms, 2015. doi:10.1145/2833179.2833186

    2015

  17. [25]

    A. N. Ziogas, G. Kwasniewski, T. Ben-Nun, T. Schneider, T. Hoefler, Deinsum: Practically I/O optimal multi-linear alge- bra, in: SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, 2022, pp. 1–15. doi:10.1109/SC41404.2022.00030

    2022 arXiv

  18. [26]

    Anderson, Z

    E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, et al., LAPACK Users’ Guide: Third Edition, 1999

    1999

  19. [27]

    Dongarra, P

    J. Dongarra, P. Luszczek, ScaLAPACK, 2011, pp. 1773–1775. doi:10.1007/978-0-387-09766-4_151

    2011 doi

  20. [28]

    Dongarra, I

    J. Dongarra, I. Duff, M. Gates, A. Haidar, S. Hammarling, et al., A proposed API for batched basic linear algebra subprograms, MIMS EPrint 2016.25, The University of Manchester (2016)

    2016

  21. [29]

    E. Chan, M. Heimlich, A. Purkayastha, R. van de Geijn, Col- lective communication: theory, practice, and experience, Con- currency and Computation: Practice and Experience 19 (2007) 1749–1783. doi:https://doi.org/10.1002/cpe.1206

    2007 doi

  22. [30]

    Dagum, R

    L. Dagum, R. Menon, OpenMP: An industry standard API for shared-memory programming, IEEE Computational Science and Engineering 5 (1998) 46–55. doi:10.1109/99.660313

    1998 doi

  23. [31]

    J. M. Perez, V. Beltran, J. Labarta, E. Ayguadé, Improving the integration of task nesting and dependencies in OpenMP, in: IEEE International Parallel and Distributed Processing Sympo- sium (IPDPS), 2017, pp. 809–818. doi:10.1109/IPDPS.2017.69

    2017 doi

  24. [32]

    J. D. McCalpin, Memory bandwidth and machine balance in current high performance computers, IEEE Computer Soci- ety Technical Committee on Computer Architecture Newsletter (1995) 19–25

    1995

  25. [33]

    P. J. Martinez-Ferrer, T. Arslan, V. Beltran, Improving the per- formance of classical linear algebra iterative methods via hybrid parallelism, Journal of Parallel and Distributed Computing 179 (2023) 104711. doi:10.1016/j.jpdc.2023.04.012

    2023 doi

  26. [34]

    Gupta, A

    S. Gupta, A. Agrawal, K. Gopalakrishnan, P. Narayanan, Deep learning with limited numerical precision, in: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, 2015, pp. 1737–1746

    2015

  27. [35]

    Kalamkar, D

    D. Kalamkar, D. Mudigere, N. Mellempudi, D. Das, K. Banerjee, et al., A study of bfloat16 for deep learning training (2019). doi:10.48550/arxiv.1905.12322

    2019 doi arXiv

  28. [36]

    Rau, IEEE 754-based half-precision floating-point library (Version 2.2) [Computer software], SourceForge, 2023

    C. Rau, IEEE 754-based half-precision floating-point library (Version 2.2) [Computer software], SourceForge, 2023. URL: https://half.sourceforge.net. 16

    2023

  29. [2016]

    doi:10.5555/3014904.3015017

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.