Pith. sign in

REVIEW 4 major objections 5 minor 5 cited by

Recipes for Pre-training LLMs with MXFP8

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MXFP8-E4M3 pre-training with a round-up scaling rule matches BF16 accuracy on models up to 8B parameters trained on up to 15T tokens.

desk verdict Solid, practical recipe paper: MXFP8-E4M3 with round-up scaling matches BF16 in pre-training, but the accuracy claim rests on Hopper emulation, not native Blackwell execution. read the letter →

arxiv 2506.08027 v2 pith:I5E5PQ66 submitted 2025-05-30 cs.LG cs.AIcs.DC

Asit Mishra , Dusan Stosic , Simon Layton , Paulius Micikevicius This is my paper
classification cs.LGcs.AIcs.DC
keywords MXFP8low-precisionpre-trainingE4M3microscalingformatsquantizationlargelanguagemodelsBlackwellscalefactorrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Every major GEMM tensor in a large language model can be stored and computed in 8-bit MXFP8 without degrading the result, provided two choices are made: use the E4M3 encoding for weights, activations, and activation gradients alike, and convert the per-block scale factor by rounding its exponent up to a power of two rather than down. The paper demonstrates this by pre-training an 8B-parameter model on 15T tokens and showing that validation perplexity stays within about 0.5% of a BF16 baseline throughout training and that downstream scores on MMLU and nine reasoning tasks match. This matters because earlier FP8 recipes required per-tensor scaling, kept some layers in high precision, and used E5M2 for gradients; the MXFP8 recipe removes those constraints while running about 2x faster than BF16 on Blackwell hardware. A mixture-of-experts model with 16B total parameters trained on 1T tokens also ends within 0.1% of its BF16 loss.

What carries the argument

The central object is the MXFP8-E4M3 datatype, an 8-bit floating-point format (1 sign, 4 exponent, 3 mantissa bits) stored in blocks of 32 elements that share one power-of-two scale factor $X$, encoded as an 8-bit integer exponent (UE8M0). The argument turns on Algorithm 1, which computes $X$ from the block's maximum absolute value, takes the ceiling of its base-2 logarithm, and clamps to the representable exponent range, so that after scaling by $X^{-1}$ no value exceeds the largest representable number. The recipe also specifies which tensors are quantized: all transformer GEMM weights, activations, and activation gradients in E4M3, while attention BMMs, softmax, residual adds, embeddings, and the output projection remain in high precision.

What would settle it

Train the same 8B model on 15T tokens on native Blackwell MXFP8 hardware using the paper's E4M3-all recipe and round-up conversion; if validation perplexity or downstream task scores deviate from the BF16 baseline by more than the reported margins (roughly 0.5% perplexity) or diverge from the emulation, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that MXFP8 training needs only two ingredients to match BF16: the E4M3 8-bit encoding for all quantized tensor types, and a conversion algorithm that rounds the per-block shared scale factor upward to the next power of two so that no scaled value overflows the destination format. The difference from the earlier specification is a single 'ceil' instead of round-down in computing the scale exponent, and the paper shows this change is what prevents the scaled values from overflowing and introducing extra quantization noise. With these choices, an 8B model trained on 15T tokens tracks the BF16 validation perplexity curve within 0.5% and matches downstream task scores, and a 16B-total/2.5B-active MoE model trained on 1T tokens finishes within 0.1% of its BF16 loss.

Load-bearing premise

The numerical evidence comes from emulating MX operations by converting BF16 tensors to MXFP8 and back on prior-generation GPUs, so the recipe's accuracy and throughput claims depend on that emulation matching the behavior of real MX Tensor Cores on Blackwell hardware.

Editorial extensions

If this is right

  • All transformer GEMM tensors can be quantized to MXFP8, removing the need to keep any transformer layer in high precision as per-tensor FP8 recipes required.
  • E4M3 suffices for activation gradients when per-block scaling is available, so the E5M2 encoding used for gradients in earlier FP8 training is unnecessary.
  • End-to-end MXFP8 pre-training on Blackwell runs at roughly 2x the throughput of BF16 with equal accuracy, making it a faster option for large runs.
  • The round-up conversion rule is recommended for MX-formats generally, so the same design choice should govern MXFP6 and MXFP4 pre-training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that, if the emulation is faithful, pre-training frameworks can develop and tune MXFP8 numerics in software before committing to the Blackwell hardware path.
  • The round-up scaling principle likely becomes even more important in MXFP6 and MXFP4, where dynamic range is tighter and a single overflowed value in a block could corrupt more of the block's information; this is a natural testable extension of the paper's conversion recipe.
  • The recipe stores row- and column-quantized copies of each tensor, doubling storage for those tensors; the paper flags this as future work, and a conversion scheme needing only one copy would directly improve the memory-accuracy trade-off.
  • The same overflow-avoidance logic should apply to block-scaled FP8 pre-training outside the MX format, since that approach also relies on per-block scales and would suffer from round-down overflow noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a recipe for pre-training LLMs using MXFP8 numerics, centered on two choices: using E4M3 encoding for all quantized tensors (weights, activations, gradients) and a specific round-up scale-factor conversion algorithm for converting high-precision values to MXFP8. The authors claim that this recipe allows MXFP8 pre-training to match BF16 pre-training in validation perplexity and downstream task accuracy for models up to 8B parameters trained on up to 15T tokens, and that Blackwell hardware performs MXFP8 GEMMs at 2x the throughput of BF16. The central evidence is a set of large-scale experiments on Hopper GPUs where MX operations are emulated in software, along with smaller ablations that motivate the datatype and rounding choices.

Significance. If the claims hold, the paper provides a practical and simple recipe for full 8-bit pre-training at parity with BF16, which would be a significant efficiency result for LLM training. The paper is clearly written and the ablation studies (datatype choice, scale rounding mode) are well structured, covering model sizes from 843M to 8B and including a MoE validation in the appendix. The choice of E4M3 for gradients is a useful empirical result, and the proposed scale-rounding fix to the OCP v1.0 conversion is concrete and falsifiable. However, the evidence path is weakened by the reliance on software emulation rather than native hardware, the absence of multiple seeds or error bars, and an unsupported throughput claim. The core recipe may be correct, but the manuscript as written does not fully substantiate the transferability of the numerical equivalence to actual Blackwell systems.

major comments (4)
  1. [Section 3.1 and Appendix A.1] The central claim that MXFP8 matches BF16 is established entirely through emulation: 'MX operations were simulated by converting BF16 inputs to MXFP8 and back to BF16 prior to GEMM operations,' and Appendix A.1 confirms that scale-factor rounding is emulated in bit-space. No native Blackwell training run or even a GEMM-level comparison against hardware MX kernels (e.g., cuBLAS/cuDNN MXFP8 paths) is provided. The emulation may faithfully reproduce the software conversion, but it does not establish that the real MX Tensor Core path on Blackwell has identical behavior for scale-factor rounding, saturation, NaN/Inf handling, or accumulation order. Since the equivalence claim is the paper's headline result, this is a load-bearing gap; the authors should either report native Blackwell results or provide a careful validation that the emulation is numerically equivalent to hardware.
  2. [Section 3.1 and Figure 2] The 'match' criterion is a self-defined 0.50% difference in validation perplexity, and all comparisons appear to be single runs without multiple seeds or error bars. LLM pre-training runs are known to have run-to-run variance that can exceed 0.5% in perplexity, and downstream task scores such as MMLU are noisy; without variance estimates or a statistical test, the claim that MXFP8 'matches' BF16 is not robustly supported. At minimum, the authors should report the number of seeds and provide the individual run curves or error bars for the main 8B/15T comparison.
  3. [Section 3.1] The paper states that 'On Blackwell GPU-based systems, MXFP8 has 2× higher throughput than BF16 making end-to-end MXFP8 pre-training faster than BF16 pre-training,' but no Blackwell experiments are reported; the training described in this section was performed on 3072 Hopper GPUs with emulated MX operations. The 2x figure appears to derive from hardware specifications (Table 1) rather than a measured end-to-end result. Since faster training is a key motivation for the recipe, this claim needs substantiation with actual Blackwell measurements or should be explicitly presented as a projected hardware capability rather than an empirical result.
  4. [Section 3.1 vs. Appendix A.5 (Table 2)] The hyperparameters for the main 8B, 15T-token experiment in Section 3.1 conflict with the configuration table in Appendix A.5. Section 3.1 states a sequence length of 8192, batch size 768, initial learning rate 6e-4, and final learning rate 6e-6, whereas Table 2 lists sequence length 4096, batch size 1024, initial LR 3e-4, and final LR 3e-7 for the 8B model. This is a direct inconsistency in the reported setup of the headline experiment and makes the results non-reproducible as written. The authors should clarify which settings apply to the 15T run, and ensure Table 2 is corrected or annotated to cover all configurations.
minor comments (5)
  1. [Figure 5] The label 'experiments stopped early' in Figure 5b is not explained in the text or caption; please clarify why the OCP-based runs were stopped and how this affects the interpretation of the divergence.
  2. [Section 3.3 / Algorithm 1] Algorithm 1 uses 'ceil(expXfloat)' but the text describes rounding-up toward positive infinity; the notation is clear, but the pseudocode does not mention handling of subnormal inputs for the scale factor (e.g., clamping at 2^-127), which is described only in Appendix A.1. Consider aligning Algorithm 1 with the full rounding procedure.
  3. [Section 3.2.1] The phrase 'in contrast to per-tensor [9] or per-row quantization [20] for FP8' seems to attribute per-tensor scaling to [9] (Nemotron-H) and per-row scaling to [20] (Llama 3); please verify these references, as the associated scaling schemes may not be described in the cited sources in exactly this way.
  4. [Section 3.1] The model description includes 'GQA group size is 8' and 'KV-channels count is 128', but these terms are not defined for the general reader; a brief definition or pointer to a prior description would aid reproducibility.
  5. [Figure 3] The diagram labels 'To next layer', 'To optimizer for master-weights update', and the transpose/quantize flows are not fully described in the text; a short walkthrough of the FPROP/DGRAD/WGRAD paths would make the recipe more actionable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the MXFP8-vs-BF16 equivalence is an external empirical benchmark, and self-citations are background rather than load-bearing.

full rationale

The paper's central claim—that MXFP8-E4M3 with a round-up scale conversion algorithm matches BF16 pre-training—is established empirically against an external BF16 baseline, not derived from the recipe choices or from the paper's own definitions. The ablations in Figures 4 and 5 choose E4M3 for all tensor types and the round-up scale factor, and the subsequent 8B/15T run (Figure 2) and MoE run (Figure 6) confirm equivalence on settings not used to fit the recipe. The self-citations (OCP MX specification, prior MXFP papers, FP8 format papers) supply background, format definitions, and prior hardware context; they are not invoked as evidence for the headline equivalence, which rests on the paper's own controlled comparisons. The software-emulation caveat is a validity and transferability concern about matching real Blackwell hardware, not a circularity concern, because the emulation was not constructed so as to force BF16-matching outcomes. No load-bearing step reduces by construction to a fitted parameter, a self-citation chain, or a renamed input.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new formal entities, particles, or forces. Its central claim rests on empirical recipe choices rather than fitted parameters. The only explicit numerical threshold used to declare success is the 0.50% match tolerance. The assumptions above are the load-bearing premises that connect the emulated experiments to the stated conclusions about real hardware and real-world pre-training.

free parameters (1)
  • match tolerance = 0.50% validation perplexity difference
    Threshold used to define 'matches BF16'; chosen by the authors without statistical justification, and directly used to support the central claim of equivalence.
assumptions (5)
  • domain assumption Emulated MXFP8 conversion on Hopper GPUs faithfully represents native Blackwell MX execution.
    Section 3.1: 'MX operations were simulated by converting BF16 inputs to MXFP8 and back to BF16 prior to GEMM operations.' If emulation differs from hardware, the numerical and throughput claims may not transfer.
  • domain assumption The proprietary high-quality dataset subset is representative for LLM pre-training.
    Appendix A.5 states the 1T and 300B token datasets are subsets of the 17T dataset from [35]; data quality and distribution are assumed sufficient to draw general conclusions.
  • domain assumption Validation perplexity and downstream task scores (MMLU, 9 reasoning tasks) are reliable proxies for model quality.
    Used throughout Section 3.1 to conclude BF16 equivalence.
  • domain assumption Single training run per configuration is sufficient to detect meaningful differences.
    All comparisons in Figures 2, 4, and 5 are single runs without seeds or error bars.
  • standard math Rounding up the scale factor minimizes quantization noise by preventing overflow after scaling.
    Section 3.3: the claim that rounding up X ensures no V/X exceeds the destination max is a logical consequence of the definition, but the optimality for training is an empirical hypothesis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recipes for Pre-training LLMs with MXFP8." pith.science (2026). https://pith.science/paper/I5E5PQ66

@misc{pith2026250608027,
  author       = {Pith},
  title        = {Pith review of: Recipes for Pre-training LLMs with MXFP8},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5E5PQ66}},
  note         = {Machine review of arXiv:2506.08027}
}
read the original abstract

Using fewer bits to represent model parameters and related tensors during pre-training has become a required technique for improving GPU efficiency without sacrificing accuracy. Microscaling (MX) formats introduced in NVIDIA Blackwell generation of GPUs represent a major advancement of this technique, making it practical to combine narrow floating-point data types with finer granularity per-block scaling factors. In turn, this enables both quantization of more tensors than previous approaches and more efficient execution of operations on those tensors. Effective use of MX-formats requires careful choices of various parameters. In this paper we review these choices and show how MXFP8-E4M3 datatype and a specific number conversion algorithm result in training sessions that match those carried out in BF16. We present results using models with up to 8B parameters, trained on high-quality datasets of up to 15T tokens.

Figures

Figures reproduced from arXiv: 2506.08027 by the authors.

Figure 1
Figure 1. A single MXFP block (in green box) and interpretation of MXFP format. 2 Microscaling format support in NVIDIA Blackwell Background: An MX-format is specified by the block size K, a shared scaling factor per block, X, and the data-type of elements in the block. A block is a contiguous sequence of K elements. K = 32 for all MX types. The data-type of X is UE8M0 and X encodes either NaN or any power-of-two1 value in th… view at source ↗
Figure 2
Figure 2. Pre-training a 8B LLM on 15T tokens. Top: Training behavior of BF16 vs MXFP8. Bottom: Comparing BF16, FP8 and MXFP8’s downstream task scores on MMLU and a set of 9 reasoning tasks. MXFP8 numerics use our proposed rounding method and E4M3 for all quantized tensors. 3 Pre-training with MXFP8 In this section we review MXFP8 training results, training recipe, and the algorithm for converting to MXFP8 used for training. … view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Pre-training loss curves comparing E4M3 and E5M2 when used across different tensor types: weights (W), activation (A) and gradients (G). The inset shows a zoomed-in view of the loss at the end of training. 3.2.2 FP8 Encoding Choice Our MXFP8 recipe uses E4M3 encoding f…
Figure 5
Figure 5. Figure 5: Comparing scale factor rounding modes suggested in the OCP v1.0 specification and our [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: MXFP8 versus BF16 for a MoE model A.5 Model configurations We conduct numerical experiments on LLM pre-training with variants of Nemotron-4 [8] models. Training and model details are described below. The 1T and 300B tokens dataset are a subset of the 17T data set discu…

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Heterogeneity-Aware Microscaling for Efficient Low-Bit LLM Inference

    cs.AR 2026-08 conditional novelty 6.0 of 10

    AdaMX puts per-block format and enhancement metadata into spare MX exponent bits, recovering 83% of MXFP4's commonsense accuracy loss at the same 4-bit storage budget.

  2. Stable FP4 Training via Transposition-Invariant Block Quantization

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Transposition-invariant 2D block FP4 quantization plus truncation-free scaling and stochastic rounding enables stable end-to-end FP4 LLM training within ~1% of BF16.

  3. SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales

    cs.LG 2026-07 conditional novelty 6.0 of 10

    SOAP and Muon, stabilized by per-step QR eigenbasis updates and KL-Shampoo covariance accumulation, beat AdamW on large-batch LLM pretraining up to 100M-token batches.

  4. ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    ACRL stabilizes FP8-quantized LLM reinforcement learning by adaptively scaling token-level importance ratios with the measured training-inference discrepancy.

  5. A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A hybrid-granularity FP8 training recipe achieves near-lossless accuracy versus BF16 on reasoning benchmarks with up to 22% faster training and 14% lower memory.

Reference graph

Works this paper leans on

40 extracted references · 12 canonical work pages · cited by 5 Pith papers

  1. [1]

    Ocp microscaling (mx) specification

    Bita Darvish Rouhani, Nitin Garegrat, Tom Savell, Ankit More, Kyung-Nam Han, Ritchie Zhao, Mathew Hall, Jasmine Klar, Eric Chung, Yuan Yu, Michael Schulte, Ralph Wittig, Ian Bratt, Nigel Stephens, Jelena Milanovic, John Brothers, Pradeep Dubey, Marius Cornea, Alexander Heinecke, Andres Rodriguez, Martin Langhammer, Summer Deng, Maxim Naumov, Paulius Micik...

    work page 2023

  2. [2]

    URL https://resources.nvidia.com/ en-us-blackwell-architecture

    Nvidia Blackwell Architecture Technical Brief. URL https://resources.nvidia.com/ en-us-blackwell-architecture

    work page

  3. [3]

    Microscaling data formats for deep learning, 2023

    Bita Darvish Rouhani, Ritchie Zhao, Ankit More, Mathew Hall, Alireza Khodamoradi, Summer Deng, Dhruv Choudhary, Marius Cornea, Eric Dellinger, Kristof Denolf, Stosic Dusan, Venmugil Elango, Maximilian Golub, Alexander Heinecke, Phil James-Roxby, Dharmesh Jani, Gaurav Kolhe, Martin Langhammer, Ada Li, Levi Melnick, Maral Mes- makhosroshahi, Andres Rodrigue...

    arXiv 2023

  4. [4]

    With shared microexpo- nents, a little shifting goes a long way, 2023

    Bita Rouhani, Ritchie Zhao, Venmugil Elango, Rasoul Shafipour, Mathew Hall, Maral Mes- makhosroshahi, Ankit More, Levi Melnick, Maximilian Golub, Girish Varatkar, Lei Shao, Gau- rav Kolhe, Dimitry Melts, Jasmine Klar, Renee L’Heureux, Matt Perry, Doug Burger, Eric Chung, Zhaoxia Deng, Sam Naghshineh, Jongsoo Park, and Maxim Naumov. With shared microexpo- ...

    arXiv 2023

  5. [5]

    Dally, and Brucek Khailany

    Steve Dai, Rangharajan Venkatesan, Haoxing Ren, Brian Zimmer, William J. Dally, and Brucek Khailany. Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference, 2021. URL https://arxiv.org/abs/2102.04503

    arXiv 2021

  6. [6]

    IEEE Std 754-2008 , pages 1–70, 2008

    Ieee standard for floating-point arithmetic. IEEE Std 754-2008 , pages 1–70, 2008. doi: 10.1109/IEEESTD.2008.4610935

    work page arXiv 2008

  7. [7]

    Fp8 formats for deep learning, 2022

    Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellem- pudi, Stuart Oberman, Mohammad Shoeybi, Michael Siu, and Hao Wu. Fp8 formats for deep learning, 2022. URL https://arxiv.org/abs/2209.05433

    arXiv 2022

  8. [8]

    Nemotron-4 15b technical report, 2024

    Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subrama- nian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, Vibhu Jawa, Jiwei Liu, Ameya Mahabaleshwarkar, Osvald Nitski, Annika Brundyn, James Maki, Miguel Martinez, Jiaxuan You, John Kamalu, Patrick LeGresley, Denys Fridman, Jared Casper, Ashwath...

    arXiv 2024

Show all 40 references
  1. [9]

    Nemotron-h: A family of accurate and efficient hybrid mamba-transformer models, 2025

    NVIDIA, :, Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, Andrew Tao, Anna Shors, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta,...

    2025 arXiv

  2. [10]

    Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020. URL https://arxiv.org/abs/1909.08053

    2020 arXiv

  3. [11]

    Measuring massive multitask language understanding, 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021. URL https: //arxiv.org/abs/2009.03300

    2021 arXiv

  4. [12]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018

    2018 arXiv

  5. [13]

    Race: Large-scale reading comprehension dataset from examinations

    Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683, 2017

    2017 arXiv

  6. [14]

    Piqa: Reasoning about physical commonsense in natural language, 2019

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language, 2019. URL https://arxiv.org/abs/ 1911.11641

    2019 arXiv

  7. [15]

    Winogrande: An adversarial winograd schema challenge at scale, 2019

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale, 2019. URL https://arxiv.org/abs/1907. 10641

    2019

  8. [16]

    Hellaswag: Can a machine really finish your sentence?, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?, 2019. URL https://arxiv.org/abs/1905.07830

    2019 arXiv

  9. [17]

    Can a suit of armor conduct electricity? a new dataset for open book question answering, 2018

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering, 2018. URL https: //arxiv.org/abs/1809.02789

    2018 arXiv

  10. [18]

    Socialiqa: Com- monsense reasoning about social interactions, 2019

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Com- monsense reasoning about social interactions, 2019. URL https://arxiv.org/abs/1904. 09728

    2019

  11. [19]

    CommonsenseQA: A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Lang...

    2019 doi

  12. [20]

    The llama 3 herd of models, 2024

    Aaron Grattafiori et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/ 2407.21783

    2024 arXiv

  13. [21]

    8-bit numerical formats for deep neural networks, 2022

    Badreddine Noune, Philip Jones, Daniel Justus, Dominic Masters, and Carlo Luschi. 8-bit numerical formats for deep neural networks, 2022. URL https://arxiv.org/abs/2206. 02915

    2022

  14. [22]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

  15. [23]

    Ocp 8-bit floating point specification (ofp8)

    Paulius Micikevicius, Stuart Oberman, Pradeep Dubey, Marius Cornea, Andres Ro- driguez, Ian Bratt, Richard Grisenthwaite, Norm Jouppi, Chiachen Chou, Amber Huff- man, Michael Schulte, Ralph Wittig, Dharmesh Jani, and Summer Deng. Ocp 8-bit floating point specification (ofp8). ...

    2023

  16. [24]

    Smoothquant: Accurate and efficient post-training quantization for large language models,

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models,

  17. [25]

    Qserve: W4a8kv4 quantization and system co-design for efficient llm serving, 2025

    Yujun Lin, Haotian Tang, Shang Yang, Zhekai Zhang, Guangxuan Xiao, Chuang Gan, and Song Han. Qserve: W4a8kv4 quantization and system co-design for efficient llm serving, 2025. URL https://arxiv.org/abs/2405.04532

    2025 arXiv

  18. [26]

    Gptq: Accurate post-training quantization for generative pre-trained transformers, 2023

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers, 2023. URL https://arxiv.org/abs/ 2210.17323

    2023 arXiv

  19. [27]

    Awq: Activation-aware weight quantization for llm compression and acceleration, 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration, 2024. URL https://arxiv.org/abs/2306.00978

    2024 arXiv

  20. [28]

    Scaling fp8 training to trillion-token llms, 2025

    Maxim Fishman, Brian Chmiel, Ron Banner, and Daniel Soudry. Scaling fp8 training to trillion-token llms, 2025. URL https://arxiv.org/abs/2409.12517. 10

    2025 arXiv

  21. [29]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innova- tion

    Meta AI. The llama 4 herd: The beginning of a new era of natively multimodal ai innova- tion. https://ai.meta.com/blog/llama-4-multimodal-intelligence/ , April 2025. Accessed 12 May 2025

    2025

  22. [30]

    Training llms with mxfp4, 2025

    Albert Tseng, Tao Yu, and Youngsuk Park. Training llms with mxfp4, 2025. URL https: //arxiv.org/abs/2502.20586

    2025 arXiv

  23. [31]

    Optimizing large language model training using fp4 quantization, 2025

    Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, and Peng Cheng. Optimizing large language model training using fp4 quantization, 2025. URL https://arxiv.org/abs/2501.17116

    2025 arXiv

  24. [32]

    Transformer engine

    Nvidia. Transformer engine. https://github.com/NVIDIA/TransformerEngine/

  25. [33]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/ 2402.03300

    2024 arXiv

  26. [34]

    Understanding warmup-stable-decay learning rates: A river valley loss landscape perspective, 2024

    Kaiyue Wen, Zhiyuan Li, Jason Wang, David Hall, Percy Liang, and Tengyu Ma. Understanding warmup-stable-decay learning rates: A river valley loss landscape perspective, 2024. URL https://arxiv.org/abs/2410.05192

    2024 arXiv

  27. [35]

    Nvidia, :, Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H. Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, Sirshak Das, Ayush Dattagupta, Olivier Delalleau, Leon Derczynski, Yi Dong, Daniel Egert, Ellie Evans, Aleksander F...

    2024 arXiv

  28. [38]

    Compute the decoding scale as: decode_scale = block_amax/destmax

  29. [39]

    if decode_scale is below 2−127, then set it to 2−127 (which is the smallest value representable in UE8M0)

  30. [40]

    By construction amax/destmax never exceeds 2127 (which is the largest value representable in UE8M0) with FP8, FP6 or FP4 formats

    For all other values that are not powers of 2, round-up to the closest representable UE8M0 value. By construction amax/destmax never exceeds 2127 (which is the largest value representable in UE8M0) with FP8, FP6 or FP4 formats. The above computations are done in the bit-space ...

  31. [2024]

    URL https://arxiv.org/abs/2211.10438

    arXiv

  32. [2025]

    URL https://arxiv.org/abs/2412.19437

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.