pith. sign in

arxiv: 2606.04028 · v1 · pith:OP4LH2NNnew · submitted 2026-06-01 · 💻 cs.LG

Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning

Andrew Fitzgibbon , Christoph M. Wintersteiger , Jeffrey Sarnoff This is my paper

Pith reviewed 2026-06-28 15:53 UTC · model grok-4.3

classification 💻 cs.LG
keywords IEEE P3109floating-point formatsmachine learning arithmeticlow-precision representationstochastic roundingclosed extended realskappa-approximationexception-free operations
0
0 comments X

The pith

IEEE P3109 defines a family of parameterized binary floating-point formats that decode to closed extended reals for exception-free machine learning operations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper details how the P3109 draft standard creates a flexible set of low-bit floating-point formats by parameterizing width, precision, signedness, and the inclusion of infinities. These formats convert values to the closed extended reals so that arithmetic operations invoke only real-number rules after NaNs and infinities are handled separately. The result is a collection of exception-free operations that report issues only through return values, plus uniform definitions for block operations that share a scale factor and a scale-invariant way for vendors to specify approximate implementations.

Core claim

The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning. These formats allow efficient and consistent representation of values in a small number of bits. The defined formats are parameterized over width and precision in bits, signedness, and the presence of infinities. Operations are defined by decoding floating-point values to the set of closed extended reals. Explicit treatment of NaN and infinite operands ensures that only real arithmetic is invoked in operation definitions. Extensive rounding and saturation modes are defined; stochastic rounding is included. Operations are excep

What carries the argument

Parameterized binary floating-point formats decoded to closed extended reals, with kappa-approximation as the measure for approximate implementations.

If this is right

  • Formats support consistent low-bit representations for machine learning workloads without vendor-specific exceptions.
  • Exception-free operations improve throughput by eliminating the need for separate exception handling paths.
  • Block operations with a shared scale factor can be implemented uniformly from the scalar definitions.
  • Vendors gain a standardized way to specify and compare approximate implementations using kappa-approximation.
  • Formal specification allows mechanical verification of standard functions and arithmetic properties.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The uniform block-scale treatment could simplify software libraries that already use blocked quantization for neural-network inference.
  • Kappa-approximation may serve as a common yardstick when comparing hardware from different vendors on the same P3109 parameters.
  • Because operations stay within real arithmetic after NaN handling, the design may reduce the surface area for numerical surprises in trained models.
  • The exception-free contract could encourage hardware designers to expose the full set of rounding modes without performance penalty.

Load-bearing premise

Explicit treatment of NaN and infinite operands ensures that only real arithmetic is invoked in operation definitions, enabling exception-free operations communicated only through return values.

What would settle it

A concrete implementation of any P3109 format that requires non-real arithmetic steps when NaN or infinity operands appear, or that produces inconsistent results for the same input across different width or precision parameters.

Figures

Figures reproduced from arXiv: 2606.04028 by Andrew Fitzgibbon, Christoph M. Wintersteiger, Jeffrey Sarnoff.

Figure 2
Figure 2. Figure 2: Auxiliary function RoundAway takes the rounding mode and truncated fraction η = S − ⌊S⌋ to determine whether to round away from zero. Stochastic rounding modes are explicitly supplied with random bits 0 ≤ R < 2N . stochastic rounding [13]. The precise behavior of these modes is defined in the function ωRoundToPrecision : R ω 7→ R ω, ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: The standards text for the behavior of Divide in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Kappa-approximation. An approximate implementation [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Operation mapping. An example showing how system-supplied [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Specification of the ωDecode and ωEncode operations, lightly paraphrased from the standard (which uses auxiliary operations ωDecodeExternal and ωDecodeAux, and more carefully extracts bitwidth K, precision P, and bias B from f). In contrast to IEEE-754, explicit integer arithmetic is used to extract the exponent and significand fields, but implementing hardware does not need to implement integer arithmetic… view at source ↗
read the original abstract

The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning. These formats allow efficient and consistent representation of values in a small number of bits. The defined formats are parameterized over width and precision in bits, signedness, and the presence of infinities. Operations are defined by decoding floating-point values to the set of closed extended reals: the reals augmented with positive and negative infinity and NaN (Not a Number). Explicit treatment of NaN and infinite operands ensures that only real arithmetic is invoked in operation definitions. Extensive rounding and saturation modes are defined; stochastic rounding is included. Operations are exception-free, accelerating throughput, with exceptional situations communicated through return values, e.g., NaN. Operations on blocks of values sharing a common scale factor are defined in terms of the underlying operations in a uniform manner. System vendors may describe approximate implementations via a novel scale-invariant measure, akin to units in the last place, called kappa-approximation. Standard function definitions and various other properties are mechanically verified and generated using formal specifications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The manuscript describes novel aspects of the IEEE SA P3109 draft standard, which defines a parameterized family of binary floating-point formats (over width, precision, signedness, and presence of infinities) and associated operations optimized for machine learning. Formats decode to the closed extended reals; NaN and infinity operands are handled explicitly so that only real arithmetic is invoked. Operations are exception-free (exceptions communicated only via return values such as NaN), support extensive rounding modes including stochastic rounding, define block operations with shared scale factors, introduce a scale-invariant kappa-approximation measure for approximate implementations, and include mechanically verified standard-function definitions generated from formal specifications.

Significance. If the described constructions hold, the paper supplies a formal, mechanically verified basis for consistent low-precision arithmetic in ML hardware and software. The exception-free semantics, closed-extended-real decoding, and kappa-approximation constitute concrete, reusable contributions that could improve portability and performance analysis. Explicit credit is due for the mechanical verification step, which supplies independent, reproducible evidence for the function definitions and properties.

minor comments (2)
  1. [Abstract / Introduction] The abstract and introduction should more explicitly distinguish which elements are direct restatements of the P3109 draft versus which are novel interpretive or presentational contributions of the manuscript itself.
  2. [Kappa-approximation section] Notation for the kappa-approximation (scale-invariant ulp-like measure) is introduced without a dedicated equation or definition block; a numbered definition would improve traceability when the measure is later used to characterize vendor implementations.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the detailed summary of our manuscript and the positive assessment of its significance. The recommendation for minor revision is noted. However, the report lists no specific major comments.

Circularity Check

0 steps flagged

No significant circularity; definitional content of draft standard

full rationale

The paper defines a parameterized family of binary floating-point formats and operations directly from the IEEE P3109 draft specifications. Decoding to closed extended reals, explicit NaN/inf handling, exception-free semantics, kappa-approximation, and mechanical verification are presented as definitional constructions rather than derived claims. No equations reduce a 'prediction' to a fitted input by construction, no load-bearing self-citations justify central premises, and no ansatz or uniqueness theorem is smuggled in. The content is self-contained against external benchmarks as a standard specification, with the reader's assessment of score 1.0 aligning with minor or absent circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper describes a draft standard rather than a mathematical derivation from first principles, introducing no free parameters, axioms, or invented entities in the research sense.

pith-pipeline@v0.9.1-grok · 5728 in / 1060 out tokens · 34664 ms · 2026-06-28T15:53:01.177342+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GoldenFloat: A Phi-Derived Static-Split Floating-Point Family from GF4 to GF1024 with a Lucas-Exact Integer Identity

    cs.AR 2026-06 unverdicted novelty 4.0

    GoldenFloat introduces a phi-derived rule for setting exponent and fraction widths across floating-point formats from 4 to 1024 bits, backed by open RTL generator, Lucas-exact accumulator, and FPGA implementation.

  2. An 83-Format Numeric Catalog with Bit-Exact Conformance Vectors: A Vendor-Neutral Reference for FP8, BF16, MXFP4, and Microscaling Formats

    cs.AR 2026-06 unverdicted novelty 2.0

    An 83-format numeric catalog with bit-exact conformance vectors and IEEE P3109 cross-walk serving as a vendor-neutral reference for FP8, BF16, MXFP4, and microscaling formats.

Reference graph

Works this paper leans on

17 extracted references · 2 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Cloud TPU: Machine learning accelerators for training and inference,

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers,et al., “Cloud TPU: Machine learning accelerators for training and inference,”IEEE Micro, vol. 38, no. 2, pp. 39–47, 2018

    2018

  2. [2]

    8-bit numerical formats for deep neural networks,

    B. Noune, P. Jones, D. Justus, D. Masters, and C. Luschi, “8-bit numerical formats for deep neural networks,”arXiv:2206.02915, 2022

    arXiv 2022

  3. [3]

    FP8 formats for deep learning,

    P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu, “FP8 formats for deep learning,”arXiv:2209.05433, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    OCP 8-bit floating point specification (OFP8) revision 1.0,

    P. Micikevicius, S. Oberman, P. Dubey, M. Cornea, A. Rodriguez, I. Bratt, R. Grisenthwaite, N. Jouppi, C. Chou, A. Huffman, M. Schulte, R. Wittig, D. Jani, and S. Deng, “OCP 8-bit floating point specification (OFP8) revision 1.0,” tech. rep., opencompute.org, 2023

    2023

  5. [5]

    Tesla Dojo Technology: A guide to Tesla’s configurable floating point formats and arithmetic,

    Tesla, Inc., “Tesla Dojo Technology: A guide to Tesla’s configurable floating point formats and arithmetic,” 2023

    2023

  6. [6]

    P3109 standard for arithmetic formats for machine learning

    IEEE, “P3109 standard for arithmetic formats for machine learning.” https://standards.ieee.org/ieee/3109/11165/

  7. [7]

    Interim report,

    P3109 Working Group, “Interim report,” 2026. https://github.com/ P3109/Public

    2026

  8. [8]

    OCP microscaling formats (mx) specification version 1.0

    Open Compute Project, “OCP microscaling formats (mx) specification version 1.0.” Open Compute Project Foundation, 2023

    2023

  9. [9]

    Branch cuts for complex elementary functions or much ado about nothing’s sign bit,

    W. Kahan, “Branch cuts for complex elementary functions or much ado about nothing’s sign bit,”Institute of Mathematics and its Applications Conference, 1987

    1987

  10. [10]

    Augmenting a programming language with complex arithmetic,

    W. Kahan and J. W. Thomas, “Augmenting a programming language with complex arithmetic,” tech. rep., EECS Department, University of California, Berkeley, 1991

    1991

  11. [11]

    Jax: Lax function_float_to_int_for_sort

    Google, “Jax: Lax function_float_to_int_for_sort.”https:// github.com/google/jax path jax/ src/lax/lax.py#L3934, Commit fc5960f2 (accessed 2026-02-13), 2023

    2026

  12. [12]

    Adaptive loss scaling for mixed precision training,

    R. Zhao, B. V ogel, and T. Ahmed, “Adaptive loss scaling for mixed precision training,”arXiv:1910.12385, 2019

    arXiv 1910

  13. [13]

    On stochastic rounding with few random bits,

    A. W. Fitzgibbon and S. Felix, “On stochastic rounding with few random bits,” in32nd Symp. on Comput. Arithmetic, ARITH 2025, pp. 133–140, IEEE, 2025

    2025

  14. [14]

    ImandraX documentation: The IML Language

    Imandra, Inc., “ImandraX documentation: The IML Language.” https://imandrax.dev/docs/language (accessed 2026-05-17)

    2026

  15. [15]

    Formal verification of the IEEE P3109 standard,

    C. M. Wintersteiger, “Formal verification of the IEEE P3109 standard,” 2025. https://github.com/imandra-ai/ieee-p3109

    2025

  16. [16]

    Formal verification of the IEEE P3109 standard for binary floating-point formats for machine learning,

    C. M. Wintersteiger, “Formal verification of the IEEE P3109 standard for binary floating-point formats for machine learning,” in32nd Symp. on Comput. Arithmetic, ARITH 2025, IEEE, 2025

    2025

  17. [17]

    FLoPS: Semantics, operations, and properties of P3109 floating-point representations in Lean,

    T.-C. Chang, S. Park, J. P. Lim, and S. Nagarakatte, “FLoPS: Semantics, operations, and properties of P3109 floating-point representations in Lean,”arXiv:2602.15965, 2026

    Pith/arXiv arXiv 2026