Accurate Models of NVIDIA Tensor Cores

Faizan A. Khattak; Mantas Mikaitis

REVIEW 2 major objections 5 minor 1 cited by

Accurate Models of NVIDIA Tensor Cores

T0 review · 2 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper presents software models that reproduce NVIDIA tensor-core inner-product arithmetic exactly at the bit level across nine GPU variants and all supported low-precision input formats.

desk verdict Solid reverse-engineering of NVIDIA tensor cores with shipped, bit-accurate-on-tested-inputs MATLAB models; the 'bit accurate' claim needs a distribution caveat and the unverifiable fourth carry bit should be stated up front. read the letter →

arxiv 2512.07004 v4 pith:IXE3WPGL submitted 2025-12-07 cs.MS cs.ARcs.NAmath.NA

Faizan A. Khattak , Mantas Mikaitis This is my paper

classification cs.MScs.ARcs.NAmath.NA MSC 65G5065Y04

keywords tensorcoresfloating-pointarithmeticmixedprecisionmatrixmultiplicationbit-accurateemulationroundingmodesGPUnumericalbehaviorlow-precisionformats

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to build software models of NVIDIA tensor cores, the hardware matrix multipliers in V100, A100, H100, B200, and related GPUs, that reproduce the hardware's numerical output exactly, down to the last bit. To do this, the authors first pin down microarchitectural features such as how many products are fused into one addition, how many extra bits are kept during significand alignment, and where rounding is applied. They then refine the models by comparing against real GPUs on hundreds of thousands of random inputs, repeating until model and hardware agree bit-for-bit. If the central claim is right, researchers can study and predict tensor-core arithmetic without owning the hardware, validate test vectors offline, and emulate high-precision matrix multiplication through multi-word schemes.

What carries the argument

The carrying mechanism is the combination of Generalised Numerical Feature Testing (GNFT) and Input Space Search Method (ISSM). GNFT constructs test vectors that trigger bit-level output differences to identify features like FMA size and extra alignment bits; ISSM draws random inputs, compares model against hardware, and uses mismatches to refine the model. The models themselves are built by combining fixed-point arithmetic with a custom floating-point format simulator inside a generalized block-FMA structure that can be parameterized by FMA count, extra alignment bits, rounding mode, sticky-bit handling, and interleaving pattern.

What would settle it

Run a directed search on any of the nine GPUs over small k, for example exhaustively enumerating all fp8 input pairs for k=2 or k=4, or applying a stress-test strategy that targets exponent boundaries, subnormals, and exact ties, and compare the real GPU output against the corresponding MATLAB model; a single mismatch would refute the bit-accurate claim.

Watch

Extended reading notes

Core claim

The central claim is that the numerical behavior of NVIDIA tensor cores can be captured in a parameterized software model that is bit-accurate against hardware. For each GPU and input/output format, the paper determines a small set of numerical features—FMA block size, extra alignment bits, rounding mode, accumulator width, whether products stay denormalized during accumulation, and whether the input vector is interleaved—and encodes them in a MATLAB toolbox. Randomized testing with 10^5 vectors per configuration drives an iterative refinement loop until GPU and model outputs match exactly for every run. The paper also reports specific microarchitectural findings, including that V100 aligns

Load-bearing premise

That the roughly 10^5 random test vectors drawn from a standard normal distribution represent all inputs the hardware will ever see; if tensor-core behavior differs for extreme exponents, subnormal combinations, or boundary rounding cases outside this sample, the bit-accuracy claim fails.

Editorial extensions

If this is right

Mixed-precision algorithm developers can test and debug numerical behavior without direct access to GPU hardware.
Test vectors designed to probe tensor-core numerical features can be validated against known-correct models before being run on new hardware.
Multi-word emulation of high-precision matrix multiplication can be evaluated across different GPU generations, revealing accuracy differences attributable to FMA size, alignment bits, and rounding modes.
The GNFT-plus-randomized-refinement loop can be applied to future tensor-core designs to build models quickly.
Bit-accurate models support reproducible numerical experiments and can inform ongoing standardization of low-precision arithmetic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The bit-accuracy validation samples inputs from a standard normal distribution, so the claim may not cover extreme exponent ranges, subnormal boundaries, or rare rounding ties; a directed stress test could reveal gaps the random ensemble misses.
The finding that fp8 mma on Hopper/Blackwell is computed through fp16 tensor cores with interleaved pairs implies that software using fp8 may inherit fp16 rounding behavior in unexpected ways, a connection the paper notes but does not explore.
The same methodology would translate to non-NVIDIA matrix multipliers, and the parameterized model makes cross-vendor numerical comparison straightforward, although such an extension is not attempted here.
If the models are widely adopted, they could serve as an oracle for formal verification of tensor-core arithmetic, but exhaustive proof would require coverage guarantees beyond randomized sampling.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this paper reverse-engineers NVIDIA tensor core inner products and ships MATLAB models that bit-match the hardware on every vector they tested — 30 runs of 10^5 fresh random draws across nine GPU variants. That is real evidence, and the code is public.

What is genuinely new is worth reading. On Hopper and Blackwell, fp8 via mma.sync is converted to fp16 and computed in interleaved pairs — an internal quirk that contradicts the obvious reading of the instruction set. V100 and A100 keep the largest product denormalized during alignment, which changes rounding in small but observable ways and corrects earlier published models. The H100 fp16 path is pinned to exactly two extra alignment bits. These are details that only come from careful probing of undocumented hardware, and the iterative GNFT/ISSM loop is a credible method for finding them.

The soft spots are real but narrow, and mostly about how the results are worded. The abstract's unqualified 'bit accurate' overstates the evidence: the validation distribution is standard normal, which does not exercise extreme exponent ranges, subnormal-heavy sums, or the rare overflow into the fourth carry bit. The authors admit in footnote 3 that the fourth carry bit is assumed, not verified. I think the stress-test note is right that this is a specific corner of the arithmetic path, not a generic sampling complaint. The fix is easy — say 'bit-accurate on the tested distribution' and add targeted adversarial distributions (max-magnitude products, subnormal swarms, saturated accumulations) before claiming more. Also, the abstract says 10^7 random vectors while the body says 10^5; minor, but for a validation-driven paper, consistency matters.

The circularity worry does not land. The loop refines the model on failing samples and breaks on a clean pass with new draws, so the final agreement is not on the refinement set. Fitting against a hardware oracle and then testing held-out samples is standard.

Audience: mixed-precision algorithm developers, error analysts, and anyone designing test vectors for non-standard arithmetic. The multi-word GEMM demo illustrates how the models get used. This deserves a serious referee — the empirical claims should be checked by someone with access to the same GPUs — but the referee should push for the claim-narrowing and extra corner-case validation. If the authors do that, it becomes a solid reference.

Referee Report

2 major / 5 minor

Summary. The manuscript develops MATLAB simulation models for the inner-product arithmetic of NVIDIA tensor cores on V100, A100, A2, A30, H100, H200, B200, L40S, and Ada RTX 1000 GPUs, covering fp8, fp16, bf16, and tf19 inputs with fp16/fp32 outputs. The modeling pipeline uses generalized numerical feature test vectors (GNFT) to set parameters such as FMA size, alignment bit widths, rounding modes, and accumulator size, then refines the models through an input-space search method (ISSM) that compares model output with GPU output on random inputs; the body reports 30 verification runs of 10^5 vectors (the abstract says 10^7). The paper reports bit-level agreement on all tested random configurations and demonstrates the toolbox for authenticating feature-detection test vectors and for multi-word GEMM emulation.

Significance. If the bit-accuracy result is taken at face value, the toolbox is a useful contribution: it gives the mixed-precision numerical analysis community open, inspectable models of current NVIDIA tensor cores, and the authors demonstrate a realistic application (multi-word GEMM) where architectural differences (integer bits, extra alignment bits, rounding/truncation, interleaving) change error behavior. The iterative GNFT+ISSM methodology and the reproducible GitHub code are concrete strengths. The main reservation is that the validation is empirical and covers only a narrow random input distribution; the models can be wrong in untested corners, and the paper itself flags one such corner (the fourth accumulator carry bit) that it cannot verify.

major comments (2)

[§IV-A2, footnote 3; §IV-B] The unqualified 'bit accurate' claim is not established for the fourth carry bit of the A100-family accumulator. The paper states in footnote 3 that only three extra carry bits can be detected and the fourth is 'assumed to exist by logic' because no numerical test distinguishes final normalization/truncation from an addition with the fourth carry bit set. The randomized validation in §IV-B draws 10^5 standard-normal inputs; such inputs produce aligned sums that almost never approach the top of the 2-integer-bit accumulator, so the fourth carry bit is effectively untested. If the hardware lacks this bit or performs intermediate normalization, there exist plausible inputs where the model output would differ from the GPU. Please provide a targeted test vector for this corner, or restrict the bit-accuracy claim to the tested range.
[Abstract; §III-B; §IV-B] The validation evidence is narrower than the abstract suggests. The abstract says 'randomised input vectors of 10^7 values,' while §III-B and §IV-B say 10^5 random vectors per run. More fundamentally, the standard-normal distribution used by ISSM samples a tiny fraction of the input space (the paper itself notes ~10^77 possible fp8 inputs for k=16) and is concentrated in a small exponent range; it cannot be assumed representative for extreme exponents, subnormal combinations, or rounding boundaries. The term 'semi-exhaustive comparison' is therefore misleading. I recommend either adding targeted boundary/edge cases to the validation or explicitly reporting the verified input distribution in every claim of bit accuracy.

minor comments (5)

[§IV-A1] Reference [28] is the Intel BFLOAT16 specification, not a source for a V100 tensor-core model; the phrase 'the model presented in [28, Fig. 2]' appears to be a citation error.
[§IV-A1] The author name 'Xi et al.' for reference [11] should be 'Li et al.'.
[Fig. 4 caption] Typo: 'RXT 1000' should be 'RTX 1000'.
[§IV-A5, Fig. 5 caption] The instruction string 'wgmma.mma async.aligned.m64nNk32' has inconsistent capitalization for N/K; it should be checked against the PTX ISA naming.
[§VI] Conclusion: 'simular' should be 'simulator'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the models are fitted to external GPU hardware and the bit-accuracy claim is an empirical match, not a derivation from its own inputs.

full rationale

The paper is an empirical system-identification study: model features (NFMA, neab, rounding modes, accumulator widths) are inferred from test vectors and then corrected whenever the model disagrees with GPU hardware in Algorithm 1. The ground truth is the GPU itself, so the refinement loop is a fit to an external oracle, not a self-referential derivation. The final statement that 'the MATLAB models produced results that matched the hardware outputs exactly at the bit level for every run' refers to the tested random instantiations; the paper explicitly notes the input space is enormous ('25632 ≈ 10^77 possible inputs') and that only 'a small proportion of it can be checked.' Because the loop stops on the first clean batch, the exact agreement on that batch is a termination observation rather than an out-of-sample guarantee; this is a coverage/overfitting caveat, not circularity, since a wrong model would fail the test and the comparison is against external hardware. The only internal assumption the paper flags is the fourth carry bit in the A100/H100 paths (footnote 3: 'only three extra carry bits can be detected through testing [13], and although a fourth carry bit is assumed to exist by logic...'), which is an admitted unverified model detail and hence a correctness limitation, not a circular step. Self-citations to the authors' earlier GNFT work [13] and to CPFloat [16] are methodology/software references; they are not load-bearing because the models are checked against GPU outputs and the paper even reports correcting prior conceptual models when they disagreed with hardware (e.g., V100 alignment in 25 bits rather than 24). No fitted parameter is renamed as a prediction and no equation-level self-definition occurs. Score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirically fitted tensor-core parameters (NFMA, extra alignment bits, rounding modes, interleaving) rather than on a first-principles derivation. The models' extrapolation beyond tested inputs relies on representative sampling and the assumed adder architecture.

free parameters (4)

Block FMA size NFMA = V100 fp16:4; A100/A2/A30/L40S fp16:8; H100/H200/B200 fp16:16; H100 fp8 via wgmma:32; L40S fp8:16
Inferred from generalized test vectors (Section IV-A); not documented by NVIDIA and varies by GPU and input format.
Extra alignment bits neab = V100:0; A100/A2/A30/L40S:1; H100/H200/B200:2; Ada fp8: 13 fractional bits (neab=-10)
Identified via GNFT and ISSM refinement; directly controls alignment width in the models.
Accumulator/alignment rounding mode = Truncation for most fp32-output modes; RNE for fp16-output and fp8-wgmma
Determined from model-vs-GPU mismatches; not specified by hardware documentation.
Interleaving pattern for fp8 via mma on H100/H200/B200 = Pairwise interleave; c added at end with RNE
Discovered through ISSM mismatches and NFMA=2 detection; not previously reported.

assumptions (4)

domain assumption GPU tensor cores implement a multi-term floating-point adder with a single global alignment and truncation of bits shifted out
This architectural model is assumed in Figs. 2-5 and underlies all parameter extraction; not independently verified from NVIDIA internal designs.
ad hoc to paper Random sampling from a standard normal distribution is representative of tensor-core numerical behavior
Section III-B chooses N(0,1) without theoretical justification for coverage of the full input space.
domain assumption CPFloat accurately simulates the low-precision input formats (fp8, fp16, bf16, tf19)
The models rely on CPFloat [16] for format conversion and rounding; no independent validation is provided in this paper.
domain assumption MATLAB's fma command faithfully emulates the IEEE-compliant FP64 tensor core (DMMA)
Section IV-B states DMMA is not modeled but can be emulated with fma, assuming identical FMA behavior.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Accurate Models of NVIDIA Tensor Cores." pith.science (2026). https://pith.science/paper/IXE3WPGL

@misc{pith2026251207004,
  author       = {Pith},
  title        = {Pith review of: Accurate Models of NVIDIA Tensor Cores},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IXE3WPGL}},
  note         = {Machine review of arXiv:2512.07004}
}

abstract

Matrix multiplication is a fundamental operation in both training of neural networks and inference. To accelerate matrix multiplication, Graphical Processing Units (GPUs) provide it implemented in hardware. Due to the increased throughput over the software-based matrix multiplication, the multipliers are increasingly used outside of AI, to accelerate various applications in scientific computing. However, matrix multipliers targeted at AI are at present not compliant with IEEE 754 floating-point arithmetic behaviour, with different vendors offering different numerical features. This leads to non-reproducible results across different generations of GPU architectures, at the matrix multiply-accumulate instruction level. To study numerical characteristics of matrix multipliers - such as rounding behaviour, accumulator width, normalization points, extra carry bits, and others - test vectors are typically constructed. Yet, these vectors may or may not distinguish between different hardware models, and due to limited hardware availability, their reliability across many different platforms remains largely untested. We present software models for emulating the inner product behavior of low- and mixed-precision matrix multipliers in the V100, A100, H100 and B200 data center GPUs in most supported input formats of interest to mixed-precision algorithm developers: 8-, 16-, and 19-bit floating point. These matrix multiplier models are first approximated by determining the numerical features via test vectors designed to trigger outputs sensitive to bit level differences in the implementation, followed by semi-exhaustive comparison (randomised input vectors of $10^7$ values) between the models and the actual GPU matrix multipliers - this process is repeated until the model is bit accurate.

Figures

Figures reproduced from arXiv: 2512.07004 by the authors.

**Figure 2.** A model of the inner product within the V100 GPU tensor [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** A model of the inner product within the A100 GPU tensor [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: A model of the inner product within the tensor cores of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: A model of the inner product within the tensor cores of [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: An example MATLAB listing showing how to call the GEMM [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Multi-word arithmetic experiment presented by Mary [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors
cs.CR 2026-06 unverdicted novelty 5.0 of 10

Proposes a Secure Gateway Device using passive optical splitters and coin-flip protocols to fingerprint all AI cluster I/O and eliminate covert channels without mutually trusted processors.

Reference graph

Works this paper leans on

31 extracted references · 1 linked inside Pith · cited by 1 Pith paper

[1]

Mixed-precision iterative reﬁnement using tensor cores o n GPUs to accelerate solution of linear systems,

A. Haidar, H. Bayraktar, S. Tomov, J. Dongarra, and N. J. H igham, “Mixed-precision iterative reﬁnement using tensor cores o n GPUs to accelerate solution of linear systems,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , vol. 476, no. 2243, p. 20200110, 2020

2020
[2]

Mixed precision algorithms in n umerical linear algebra,

N. J. Higham and T. Mary, “Mixed precision algorithms in n umerical linear algebra,” Acta Numerica , vol. 31, pp. 347–414, May 2022

2022
[3]

Accelerating supercomputing: AI-hardware-driven innovation for speed and efﬁciency,

J. Dongarra, J. Gunnels, H. Bayraktar, A. Haidar, and D. E rnst, “Accelerating supercomputing: AI-hardware-driven innovation for speed and efﬁciency,” in 2025 IEEE High Performance Extreme Computing Conference (HPEC) , 2025, pp. 1–7

2025
[4]

Datasheet: AMD instrinct MI355X GPU,

AMD, “Datasheet: AMD instrinct MI355X GPU,” 2025. [Onli ne]. Available: https://www.amd.com/content/dam/amd/en/documents/ instinct-tech-docs/product-briefs/amd-instinct-mi35 5x-gpu-brochure. pdf

2025
[5]

NVIDIA Blackwell architecture technical brie f,

NVIDIA, “NVIDIA Blackwell architecture technical brie f,”
[6]

OCP 8-bit ﬂoating point spec itication (OFP8),

P . Micikevicius, S. Oberman, P . Dubey, M. Cornea, A. Rodr iguez, I. Bratt, R. Grisenthwaite, N. Jouppi, C. Chou, A. Huffman, M . Schulte, R. Wittig, D. Jani, and S. Deng, “OCP 8-bit ﬂoating point spec itication (OFP8),” Open Compute Project, Tech. Rep., Jun. 2023, revis ion 1.0. [Online]. Available: https://www.opencompute.org/documents/ ocp-8-bit-float...

2023
[7]

Interim report on binary ﬂoating-point formats for mac hine learning,

“Interim report on binary ﬂoating-point formats for mac hine learning,” Tech. Rep., Nov. 2025, version 3.2. [Online]. Available: https:// github.com/P3109/Public/blob/main/Shared%20Reports/ IEEE%20WG %20P3109%20Interim%20Report%20v3.1.pdf

2025
[8]

Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers, Jul

IEEE Standard for Floating-Point Arithmetic, IEEE Std 754- 2019 (re- vision of IEEE Std 754-2008) . Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers, Jul. 2019

2019

Show all 31 references

[9]

Experimental analysis of m atrix multi- plication functional units,

B. Hickmann and D. Bradford, “Experimental analysis of m atrix multi- plication functional units,” in 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH) , Oct. 2019, pp. 116–119

2019
[10]

Nume rical behavior of NVIDIA tensor cores,

M. Fasi, N. J. Higham, M. Mikaitis, and S. Pranesh, “Nume rical behavior of NVIDIA tensor cores,” PeerJ Computer Science, vol. 7, p. e330, 2021

2021
[11]

FTTN: Feature-targeted testing for numerical pro perties of NVIDIA & AMD matrix accelerators,

X. Li, A. Li, B. Fang, K. Swirydowicz, I. Laguna, and G. Go palakr- ishnan, “FTTN: Feature-targeted testing for numerical pro perties of NVIDIA & AMD matrix accelerators,” in 2024 IEEE 24th International Symposium on Cluster , Cloud and Internet Computing (CCGrid ), 2024, pp. 39–46

2024
[12]

An SMT f ormalization of mixed-precision matrix multiplication,

B. V alpey, X. Li, S. Pai, and G. Gopalakrishnan, “An SMT f ormalization of mixed-precision matrix multiplication,” in NASA F ormal Methods . Cham: Springer Nature Switzerland, 2025, pp. 360–379

2025
[13]

Generalized methodolog y for deter- mining numerical features of hardware ﬂoating-point matri x multipliers: Part I,

F. A. Khattak and M. Mikaitis, “Generalized methodolog y for deter- mining numerical features of hardware ﬂoating-point matri x multipliers: Part I,” in 2025 IEEE High Performance Extreme Computing Conference (HPEC), Wakeﬁeld, MA, USA, Oct. 2025

2025
[14]

GPU ﬂoating-point Para noia,

K. E. Hillesland and A. Lastra, “GPU ﬂoating-point Para noia,” in ACM W orkshop on General-Purpose Computing on Graphics Processors (GP2). Los Angeles, CA, USA: ACM, Aug. 2004

2004
[15]

FPGA Paran oia: Testing numerical properties of FPGA ﬂoating point ip-cores,

X. Y . Tan, D. Boland, and G. Constantinides, “FPGA Paran oia: Testing numerical properties of FPGA ﬂoating point ip-cores,” in Reconﬁgurable Computing: Architectures, Tools and Applications , O. C. S. Choy, R. C. C. Cheung, P . Athanas, and K. Sano, Eds. Berlin, Heidelb erg: ...

2012
[16]

CPFloat: A C library for simula ting low- precision arithmetic,

M. Fasi and M. Mikaitis, “CPFloat: A C library for simula ting low- precision arithmetic,” ACM Trans. Math. Softw., vol. 49, no. 2, pp. 18:1– 18:32, Jun. 2023

2023
[17]

NVIDIA tensor core programmability, performance & precis ion,

S. Markidis, S. W. D. Chien, E. Laure, I. B. Peng, and J. S. V etter, “NVIDIA tensor core programmability, performance & precis ion,” in Proceedings of the 32nd IEEE International Parallel and Dis tributed Processing Symposium W orkshops, V ancouver, BC, Canada, Aug. 2018, pp. 522–531

2018
[18]

Accelerating non-power-of- 2 size Fourier transforms with GPU tensor cores,

L. Pisha and Ł. Ligowski, “Accelerating non-power-of- 2 size Fourier transforms with GPU tensor cores,” in Proceedings of the 2021 IEEE International Parallel and Distributed Processing Sympos ium, Portland, OR, USA, May 2021, pp. 507–516

2021
[19]

Recovering single precision a ccuracy from tensor cores while surpassing the FP32 theoretical peak per formance,

H. Ootomo and R. Y okota, “Recovering single precision a ccuracy from tensor cores while surpassing the FP32 theoretical peak per formance,” The International Journal of High Performance Computing Ap plications, vol. 36, no. 4, pp. 475–491, Jun. 2022

2022
[20]

Error analysis of matrix multi plication with narrow range ﬂoating-point arithmetic,

T. Mary and M. Mikaitis, “Error analysis of matrix multi plication with narrow range ﬂoating-point arithmetic,” SIAM J. Sci. Comput. , vol. 47, no. 4, pp. B785–B800, 2025

2025
[21]

Monotonicity of multi-term ﬂoating-poi nt adders,

M. Mikaitis, “Monotonicity of multi-term ﬂoating-poi nt adders,” IEEE Trans. Comput., vol. 73, no. 6, pp. 1531–1543, Feb. 2024

2024
[22]

Optimized fused ﬂoating-point many-term dot-product har dware for machine learning accelerators,

H. Kaul, M. Anders, S. Mathew, S. Kim, and R. Krishnamurt hy, “Optimized fused ﬂoating-point many-term dot-product har dware for machine learning accelerators,” in 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH) , 2019, pp. 84–87

2019
[23]

Intel Nervana Neural Network Processor-T (NNP-T) fus ed ﬂoating point many-term dot product,

B. Hickmann, J. Chen, M. Rotzin, A. Y ang, M. Urbanski, an d S. Avan- cha, “Intel Nervana Neural Network Processor-T (NNP-T) fus ed ﬂoating point many-term dot product,” in 2020 IEEE 27th Symposium on Computer Arithmetic (ARITH) , 2020, pp. 133–136

2020
[24]

Mixed precision block fused multiply-add: Error analysis and app lication to GPU tensor cores,

P . Blanchard, N. J. Higham, F. Lopez, T. Mary, and S. Pran esh, “Mixed precision block fused multiply-add: Error analysis and app lication to GPU tensor cores,” SIAM Journal on Scientiﬁc Computing, vol. 42, no. 3, pp. C124–C141, 2020

2020
[25]

Multi-operand ﬂoating-point addition,

A. F. Tenca, “Multi-operand ﬂoating-point addition,” in 2009 19th IEEE Symposium on Computer Arithmetic , 2009, pp. 161–168

2009
[26]

A test of a computer’s ﬂoating-point ari thmetic unit,

N. L. Schryer, “A test of a computer’s ﬂoating-point ari thmetic unit,” A T&T Bell Laboratories, Murray Hill, NJ, Murray Hill, NJ 079 74, Technical Report Computer Science Technical Report 89, Feb . 1981

1981
[27]

NVIDIA Tesla V100 GPU architecture,

NVIDIA, “NVIDIA Tesla V100 GPU architecture,” 2017. [O nline]. Available: https://images.nvidia.com/content/volta- architectur e/pdf/ volta-architecture-whitepaper.pdf

2017
[28]

BFLOA T16—hardware numerics deﬁn ition,

Intel Corporation, “BFLOA T16—hardware numerics deﬁn ition,” Avail- able at h t t p s : //s o f t w a r e .i n t e l.c o m/e n - u s/d o w n l o a d / bfloat16-hardware-numerics-definition (accessed 15 July 2020), Nov. 2018, white paper. Document number 338302-001US

2020
[29]

CUDA Binary Utilities, release 13.1,

N. Corporation, “CUDA Binary Utilities, release 13.1, ” 2025. [Online]. Available: https://docs.nvidia.com/cuda/pdf/CUDA Binary Utilities.pdf

2025
[30]

Disse cting the NVIDIA Blackwell architecture with microbenchmarks,

A. Jarmusch, N. Graddon, and S. Chandrasekaran, “Disse cting the NVIDIA Blackwell architecture with microbenchmarks,” 2025. [Online]. Available: https://arxiv.org/abs/2507.10789

2025 arXiv
[2025]

Available: h t t p s : //r e s o u r c e s.nv i d i a .c o m/ en-us-blackwell-architecture

[Online]. Available: h t t p s : //r e s o u r c e s.nv i d i a .c o m/ en-us-blackwell-architecture

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

Mixed-precision iterative reﬁnement using tensor cores o n GPUs to accelerate solution of linear systems,

A. Haidar, H. Bayraktar, S. Tomov, J. Dongarra, and N. J. H igham, “Mixed-precision iterative reﬁnement using tensor cores o n GPUs to accelerate solution of linear systems,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , vol. 476, no. 2243, p. 20200110, 2020

2020

[2] [2]

Mixed precision algorithms in n umerical linear algebra,

N. J. Higham and T. Mary, “Mixed precision algorithms in n umerical linear algebra,” Acta Numerica , vol. 31, pp. 347–414, May 2022

2022

[3] [3]

Accelerating supercomputing: AI-hardware-driven innovation for speed and efﬁciency,

J. Dongarra, J. Gunnels, H. Bayraktar, A. Haidar, and D. E rnst, “Accelerating supercomputing: AI-hardware-driven innovation for speed and efﬁciency,” in 2025 IEEE High Performance Extreme Computing Conference (HPEC) , 2025, pp. 1–7

2025

[4] [4]

Datasheet: AMD instrinct MI355X GPU,

AMD, “Datasheet: AMD instrinct MI355X GPU,” 2025. [Onli ne]. Available: https://www.amd.com/content/dam/amd/en/documents/ instinct-tech-docs/product-briefs/amd-instinct-mi35 5x-gpu-brochure. pdf

2025

[5] [5]

NVIDIA Blackwell architecture technical brie f,

NVIDIA, “NVIDIA Blackwell architecture technical brie f,”

[6] [6]

OCP 8-bit ﬂoating point spec itication (OFP8),

P . Micikevicius, S. Oberman, P . Dubey, M. Cornea, A. Rodr iguez, I. Bratt, R. Grisenthwaite, N. Jouppi, C. Chou, A. Huffman, M . Schulte, R. Wittig, D. Jani, and S. Deng, “OCP 8-bit ﬂoating point spec itication (OFP8),” Open Compute Project, Tech. Rep., Jun. 2023, revis ion 1.0. [Online]. Available: https://www.opencompute.org/documents/ ocp-8-bit-float...

2023

[7] [7]

Interim report on binary ﬂoating-point formats for mac hine learning,

“Interim report on binary ﬂoating-point formats for mac hine learning,” Tech. Rep., Nov. 2025, version 3.2. [Online]. Available: https:// github.com/P3109/Public/blob/main/Shared%20Reports/ IEEE%20WG %20P3109%20Interim%20Report%20v3.1.pdf

2025

[8] [8]

Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers, Jul

IEEE Standard for Floating-Point Arithmetic, IEEE Std 754- 2019 (re- vision of IEEE Std 754-2008) . Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers, Jul. 2019

2019