Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows NF4 and AF4 optimize the wrong objective and derives BOF4, 4-bit block-wise codebooks that minimize the true weight quantization error; signed and outlier-preserving variants reach the best data-free 4-bit perplexity.

desk verdict Useful and mostly sound incremental improvement for block-wise 4-bit LLM quantization; the MSE side is solid, but the MAE-optimality derivation in Appendix B.2.2 has a real missing term and the 'information-theoretically optimal' label oversells it. read the letter →

arxiv 2505.06653 v1 pith:RVMXILUT submitted 2025-05-10 cs.LG cs.CL

Patrick Blumenberg , Thomas Graave , Tim Fingscheidt This is my paper · ORCID
classification cs.LGcs.CL
keywords BOF4block-wisequantization4-bitlargelanguagemodelsLloyd'salgorithmoutlier-preservingQLoRApost-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing 4-bit block-wise quantizers for large language model weights, NF4 and AF4, use codebooks that minimize the quantization error of the normalized weights rather than the error of the actual network weights. This paper shows the two objectives differ: each normalized value is scaled back up by its block maximum $m$, so the true end-to-end squared error is $m^2(x-\hat{x})^2$, and it derives the codebooks that minimize that true error. The result is a family of 4-bit quantizers, BOF4 and BOF4-S, that reduce MAE and MSE relative to both baselines, with the signed-normalization variant freeing a reconstruction level by fixing only one endpoint. Adding a mixed-precision scheme that stores outlier weights in 16-bit precision (OPQ) yields the best perplexity among data-free 4-bit block-wise quantizers on the tested models. Because these quantizers need no calibration data, a better codebook translates directly to cheaper memory-efficient fine-tuning and inference.

What carries the argument

The load-bearing object is a modified Lloyd's algorithm for block-wise absmax quantization. In each maximization step, the reconstruction level of a Voronoi region is updated not to the plain mean or median of its normalized samples but to a weighted version: for MSE, $\hat{x}(\ell)=\sum_k w_k^2 x_k \big/ \sum_k w_k^2$, where $x_k$ are the normalized weights falling in the region and $w_k$ is the block maximum of the block containing each sample; for MAE, the weighted median with weights $w_k$. The paper proves these centroid conditions by deriving them from the end-to-end quantization error, and shows that the theoretical integral solution and the Monte-Carlo solution agree to about $-56\,\text{dB}$ of relative squared error. BOF4-S rests on the companion mechanism of signed absmax normalization: choosing as the block constant the sign-carrying maximum weight means only the $+1$ endpoint is a mandated reconstruction level, so the level that would otherwise be spent on $-1$ can be re-placed to lower the error.

What would settle it

Run the paper's Monte-Carlo centroid rule on the actual per-layer weights of a large pretrained LLM, quantize with the resulting per-layer codebooks, and compare MAE, MSE, and perplexity against the fixed Gaussian-derived BOF4-S codebook at the same block size. If the data-driven codebooks do not measurably beat the Gaussian-derived ones, the distributional assumption is adequate and the claimed optimality stands; if they do beat it, the claim of optimality is limited by that assumption.

Watch

Extended reading notes

Core claim

The paper claims that the optimality story behind NF4 is wrong on two counts: equal utilization of the 16 reconstruction levels is not a valid criterion for rate-distortion optimality, and the AF4 correction still minimizes the error of the normalized weights, not the error of the weights that are actually quantized. In block-wise absmax quantization a weight is reconstructed as $m\hat{x}$ from its normalized value $x$ and the block maximum $m$, so the end-to-end squared error is $m^2(x-\hat{x})^2$; the correct MSE centroid of a Voronoi region is therefore the mean of its normalized samples weighted by squared block maxima, and the correct MAE centroid is the median weighted by block maxima. Feeding these modified centroid updates into Lloyd's algorithm yields the BOF4 codebooks, and changing the normalization—using the signed absolute maximum so that only the $+1$ endpoint is a fixed reconstruction level—yields BOF4-S, which reduces the error further. The paper further claims that outlier weights violate the Gaussian assumption on which the analysis rests, and its outlier-preserving quantization (OPQ) responds by storing those outliers in bfloat16, replacing them with zero before normalization and thereby improving perplexity, especially at large block sizes.

Load-bearing premise

The codebooks are derived under the assumption that model weights are independent draws from a single zero-mean symmetric distribution, taken as Gaussian; real LLM weights have layer-dependent and outlier structure that violates this, and the paper's own OPQ mechanism exists precisely because of that failure.

Editorial extensions

If this is right

  • Data-free 4-bit block-wise quantization of LLM weights can be made lower in MAE and MSE than NF4 and AF4 at zero extra memory cost, since only the codebook values change.
  • Signed absmax normalization (BOF4-S) is a stand-alone accuracy gain that transfers to any block-wise quantizer, not just the codebooks derived here.
  • MSE-optimized codebooks generally give lower perplexity than MAE-optimized ones, making the error metric of codebook optimization itself a design choice with measurable language-modeling consequences.
  • Outlier-preserving quantization keeps perplexity low at larger block sizes, reducing the memory used by quantization constants.
  • QLoRA-style fine-tuning inherits the ranking: instruction-following and code-generation models fine-tuned on BOF4-based weights match or exceed those fine-tuned on NF4 or AF4 weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The $m^2$- and $m$-weighted centroid rules are not specific to LLM weights: any block-scaled scalar quantizer, including block-wise optimizer-state quantization, suffers the same objective mismatch, so re-deriving its codebooks with the same argument is a direct testable transfer.
  • Because the Monte-Carlo and theoretical codebooks coincide, the natural next experiment is to run the centroid rule on real per-layer weight samples; if per-layer codebooks outperform the fixed Gaussian-derived one, the 'optimal' label moves from one global codebook to a data-dependent family.
  • The OPQ threshold $q$ is fixed against a theoretical distribution of block maxima; treating it as a per-layer or per-tensor knob rather than a global hyperparameter is a testable way to trade memory against perplexity, especially for small models where OPQ showed the largest gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies 4-bit block-wise absmax quantization of LLM weights, as used in QLoRA-style fine-tuning. It proposes BOF4 and BOF4-S, codebooks obtained by a Lloyd-style EM algorithm with centroid rules that minimize the end-to-end MSE or MAE of the unnormalized weights, rather than the error of the normalized weights. It also proposes signed absmax normalization (BOF4-S), an outlier-preserving mixed-precision scheme (OPQ), and reports experiments on Llama, Qwen, and Mistral models showing lower quantization error and perplexity than NF4 and AF4. A theoretical solution and a Monte-Carlo empirical solution are both presented, with a numerical equivalence check for the MSE case.

Significance. The MSE-based BOF4-S results, if correct, would be a useful, reproducible contribution: the modified centroid derivation is nontrivial, the codebooks are published, the MSE theoretical/empirical agreement at -56 dB is a strong sanity check, and the perplexity improvements over NF4/AF4, while small, are consistent across several models. The OPQ mechanism is a practical addition for handling outliers without abandoning the Gaussian weight assumption. However, the paper's central theoretical novelty is weakened by an algebraic error in the MAE centroid derivation, which is one of the two optimality results claimed in the abstract. Since the error is local and the practical results are dominated by MSE optimization, the contribution is defensible after a substantial revision.

major comments (2)
  1. [Appendix B.2.2, Eq. (59) and Section 3.2, Eq. (7)] The derivation of the MAE centroid condition is algebraically incorrect. Starting from Eq. (57) and substituting pM(m|X∈R_l) = pM(m) A / P[X∈R_l] with A = FX(ξ(l)|M=m) - FX(ξ(l-1)|M=m), and the truncated CDF FX(x̂|M=m,X∈R_l) = (FX(x̂|M=m) - FX(ξ(l-1)|M=m))/A, one obtains ∫ m pM(m)[FX(x̂|M=m) - FX(ξ(l-1)|M=m) - A/2] dm = 0. Equation (59) omits the term -FX(ξ(l-1)|M=m). This is not a notational variant: for a uniform source on [-1,1] and region [0,1], Eq. (59) gives x̂ = -0.5, whereas the correct MAE-optimal level is the median 0.5. Consequently, the MAE-optimized codebooks in Table 6 are not proven to minimize MAE even under the paper's Gaussian model, and the claimed equivalence between the theoretical and data-driven MAE solutions (Eq. (8) vs. Eq. (59)) is unverified; Table 8 covers MSE only. The theoretical MAE derivation must be corrected or the MAE-optimality claims must be withdrawn.
  2. [Abstract, Section 1, and Section 3.2] The codebooks are described as 'information-theoretically optimal' and 'the correct, information-theoretically optimal codebook'. Appendix B.2 itself states that the modified centroid criterion establishes only local optimality of a Lloyd-algorithm fixed point. Lloyd's algorithm is not guaranteed to find a global optimum, so the unqualified 'information-theoretically optimal' overstates the result. Please replace with 'locally optimal under the stated necessary conditions' or equivalent wording.
minor comments (5)
  1. [Section 3.2, Eq. (7)] The bracket notation in Eq. (7) contains a typographical artifact ('FX(ˆx(ℓ)|M =m)]− 1/2'); the intended expression appears to be FX(ˆx(ℓ)|M =m) − 1/2 [FX(x|M =m)]ξ(ℓ)ξ(ℓ−1), matching Eq. (59). Please correct the typesetting.
  2. [Tables 1, 3, 4 and Appendix G] The reported perplexity differences between methods are small (e.g., 8.43 vs. 8.53 for Llama-3.1 8B in Table 1), and the fine-tuning experiments in Tables 3 and 4 involve stochastic training runs without variance or multiple seeds. Please report standard deviations or multiple seeds for the fine-tuning evaluations to support the ranking claims.
  3. [Appendix C, Table 8] The numerical equivalence between the theoretical and empirical centroid solutions is demonstrated only for MSE. Given the MAE derivation issue, please provide an analogous comparison for MAE or explicitly state that equivalence was verified only for MSE.
  4. [Title page, Appendix F, References] Please fix the typos: 'Insitute' should be 'Institute' (title page), 'qunaitzation' should be 'quantization' (Appendix F), 'V oronoi' should be 'Voronoi' (Section 3.2), and reference [22] should list 'AAAI' instead of 'AIII'.
  5. [Section 6, Limitations] The limitation discussion of double quantization is useful. Please also state explicitly that the OPQ threshold q was tuned on a validation split (Appendix E.2), which introduces a small data-dependent element, and briefly discuss how this interacts with the 'data-free' characterization of the method.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BOF4(-S) codebooks are derived from an external Gaussian model and validated against external baselines, not from the evaluation data.

full rationale

The central derivation is self-contained: the BOF4(-S) codebooks are obtained by minimizing the end-to-end MSE or MAE of block-wise absmax-quantized weights under an assumed zero-mean symmetric, specifically Gaussian, weight distribution, with closed-form centroid conditions in Eqs. (5) and (7) and matching Monte-Carlo estimators in Eqs. (6) and (8). The comparisons to NF4 and AF4 are made against externally fixed codebooks on fixed LLM weights, so the reported error reductions are not forced by construction. The Gaussian i.i.d. assumption is an external modeling assumption, not an import of the target result, and the empirical/theoretical equivalence check in Table 8 is a consistency test of the same objective, not a disguised fit. The OPQ hyperparameter q is selected on a validation split, but this is ordinary hyperparameter tuning rather than a fitted parameter renamed as a prediction, and it does not enter the derivation of the codebooks. No load-bearing self-citation or imported uniqueness theorem is used; Lloyd's algorithm is standard prior work. The algebraic concern about Eq. (59) raised in the skeptical note concerns the correctness of the MAE centroid derivation, which falls under mathematical validity rather than circularity, and is therefore not scored here.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central derivation assumes network weights are i.i.d. zero-mean symmetric, then Gaussian (Appendix B.1). This is an explicit distributional assumption, not proven for LLM weights; the paper compensates for its failure on outliers with OPQ. The remaining ingredients are standard Lloyd-Max theory and a known Gaussian integral. The only fitted number is the OPQ quantile q=0.95.

free parameters (2)
  • OPQ quantile q = 0.95
    Chosen via limited search over {0.9, 0.95, 0.97, 0.99} on Llama-3.1 8B WikiText-2 validation perplexity and memory overhead (Appendix E.2), then applied to all other models.
  • Block size I = 64 (main experiments); codebooks tabulated for 32, 64, 128, 256
    The codebook derivation is per block size I. I=64 is used for the headline tables; sensitivity is studied in Figures 2, 3, and 12. The choice is a design hyperparameter, not fitted to downstream metrics, but the optimal codebook depends on it.
assumptions (4)
  • domain assumption Network weights are i.i.d. samples from a zero-mean symmetric distribution, specialized to N(0,1).
    Used to derive the distribution of normalized weights and absolute block maxima in Appendix B.1 (Eqs. 10-17) and to compute all BOF4 codebooks (Section 3.2). The paper acknowledges deviations such as outliers and adds OPQ to compensate.
  • standard math The nearest-neighbor assignment is a necessary condition for optimality in the modified Lloyd algorithm.
    Invoked in Appendix B.2 to justify the EM iteration; this is standard for Lloyd-Max quantization.
  • domain assumption Within a block, the largest-magnitude weight is unique, so the discrete mass points of normalized weights are at +/-1 (absolute) or +1 (signed).
    Used to write the CDF in Eqs. (16)-(17) and in the special treatment of outermost regions. Violated only by ties, which have probability zero for continuous distributions.
  • standard math Gaussian integral identity (Owen 1980, eq. 101) used to simplify the MSE centroid for Gaussian weights.
    Used in Appendix B.2.1, Eqs. (32)-(33). This is a known integral.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations." pith.science (2026). https://pith.science/paper/RVMXILUT

@misc{pith2026250506653,
  author       = {Pith},
  title        = {Pith review of: Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RVMXILUT}},
  note         = {Machine review of arXiv:2505.06653}
}
read the original abstract

Large language models (LLMs) demand extensive memory capacity during both fine-tuning and inference. To enable memory-efficient fine-tuning, existing methods apply block-wise quantization techniques, such as NF4 and AF4, to the network weights. We show that these quantization techniques incur suboptimal quantization errors. Therefore, as a first novelty, we propose an optimization approach for block-wise quantization. Using this method, we design a family of quantizers named 4-bit block-wise optimal float (BOF4), which consistently reduces the quantization error compared to both baseline methods. We provide both a theoretical and a data-driven solution for the optimization process and prove their practical equivalence. Secondly, we propose a modification to the employed normalization method based on the signed absolute block maximum (BOF4-S), enabling further reduction of the quantization error and empirically achieving less degradation in language modeling performance. Thirdly, we explore additional variations of block-wise quantization methods applied to LLMs through an experimental study on the importance of accurately representing zero and large-amplitude weights on the one hand, and optimization towards various error metrics on the other hand. Lastly, we introduce a mixed-precision quantization strategy dubbed outlier-preserving quantization (OPQ) to address the distributional mismatch induced by outlier weights in block-wise quantization. By storing outlier weights in 16-bit precision (OPQ) while applying BOF4-S, we achieve top performance among 4-bit block-wise quantization techniques w.r.t. perplexity.

Figures

Figures reproduced from arXiv: 2505.06653 by the authors.

Figure 1
Figure 1. The blue histograms show the distributions of normalized weights pX(x) for block-wise absolute absmax normalization (left) and block-wise signed absmax normalization (right) assuming Gaussian network weights. Also shown are the resulting reconstruction levels xˆ(ℓ) (▼ fixed, ▼ optimized) and decision thresholds ξ(ℓ) (dashed lines), after minimizing the MSE(W, Q(W)) for normally distributed network weights W = (wb,i)… view at source ↗
Figure 2
Figure 2. MAE (left) and MSE (right) quantization error of our quantization methods BOF4 and BOF4-S optimized for MAE (left, *) or MSE (right, ◦) compared to the baselines NF4 and AF4 for Gaussian network weights W = (wb,i) with wb,i ∼ N (0, 1) depending on the block size I. 5 Results and Discussion Quantization Error: In [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Perplexity of Llama-3.1 8B on WikiText-2 after quantization with NF4, AF4, and our BOF4-S optimized w.r.t. MAE (left, *) or MSE (right, ◦) for different block sizes I, with and without outlier-preserving quantization (OPQ, dashed line). instruction following and code generation, respectively, and evaluated on corresponding task-specific benchmarks. For comparison, we apply LoRA fine-tuning [6] to the original, unqua… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Empiric estimation of the PDF pX, resulting from block-wise absmax normalization, in case of Gaussian network weights based on 2 29 samples x for different block sizes I. −1.0 −0.5 0.0 0.5 1.0 x 0.00 0.25 0.50 0.75 1.00 FX(x) abs. norm. signed norm [PITH_FULL_IMAGE:fi…
Figure 5
Figure 5. Figure 5: Example CDF FX(x) for absolute and signed block-wise absmax normalized Gaussian network weights x = xb,i and block size I = 8. Considering that FX,cont contains the fraction I−1 I of the probability mass, whereas +1 and −1 each occur with probability 1 2I , the CDF of …
Figure 6
Figure 6. Figure 6: Difference in perplexity on WikiText-2 of [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Illustration of OPQ outlier detection. The histogram of absolute weights |wb,i| σb of an example block b with block size I = 64 normalized to a unit standard deviation is shown in blue. Weights are identified as outliers (red hatching) iff they are greater than F −1 M …
Figure 8
Figure 8. Figure 8: Effect of outlier-preserving quantization (OPQ) on the distribution of normalized network weights. The histogram of original network weights W containing some outlier weights (red) and non-outliers (blue) is shown on the left. The normalized network weights X that are …
Figure 9
Figure 9. Figure 9: Additional OPQ memory over￾head of BOF4 (MSE) applied to Llama-3.1 8B as a fraction of the total memory required by the quantized weights after block-wise ab￾smax quantization, including the quantization constants. 2 5 2 6 2 7 2 8 2 9 2 10 2 11 block size I 8.6 8.8 Per…
Figure 11
Figure 11. Figure 11: Time to generate 1000 tokens with and without OPQ depending on the block size [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: Perplexity of Llama-3.1 8B on WikiText-2 after quantization with NF4, AF4, and our BOF4 optimized w.r.t. MAE (left, *) or MSE (right, ◦) for different block sizes I, without and with outlier-preserving quantization (OPQ, dashed line). 29 [PITH_FULL_IMAGE:figures/full…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A parametric cubic codebook with two shape parameters per group reduces quantization reconstruction error versus uniform integer and finite floating-point baselines for 1-8-bit LLM weights, and can be executed directl...

Reference graph

Works this paper leans on

27 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    OPTQ: Accurate Quantization for Generative Pre-trained Transformers,

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “OPTQ: Accurate Quantization for Generative Pre-trained Transformers,” inProc. of ICLR, Kigali, Rwanda, May 2023, pp. 1–16

    work page 2023

  2. [2]

    AWQ: Activation-Aware Weight Quantization for On-Device LLM Compression and Acceleration,

    J. Lin, J. Tang, H. Tang,et al., “AWQ: Activation-Aware Weight Quantization for On-Device LLM Compression and Acceleration,” in Proc. of MLSys, Santa Clara, CA, USA, May 2024, pp. 87–100

    work page 2024

  3. [3]

    SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, “SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,” inProc. of ICML, Honolulu, HI, USA, Jul. 2023, pp. 38 087–38 099

    work page 2023

  4. [4]

    SpinQuant: LLM Quantization with Learned Rotations,

    Z. Liu, C. Zhao, I. Fedorov, et al., “SpinQuant: LLM Quantization with Learned Rotations,” arXiv, May 2024. arXiv: 2405.16406

    arXiv 2024

  5. [5]

    QLORA: Efficient Finetuning of Quantized LLMs,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLORA: Efficient Finetuning of Quantized LLMs,” in Proc. of NIPS, New Orleans, LA, USA, Dec. 2023, pp. 10 088–10 115

    work page 2023

  6. [6]

    LoRA: Low-Rank Adaptation of Large Language Models,

    E. J. Hu, Y . Shen, P. Wallis,et al., “LoRA: Low-Rank Adaptation of Large Language Models,” in Proc. of ICLR, virtual, Apr. 2022, pp. 1–13

    work page 2022

  7. [7]

    NF4 Isn’t Information Theoretically Optimal (and That’s Good),

    D. Yoshida, “NF4 Isn’t Information Theoretically Optimal (and That’s Good),” arXiv, Jun

    work page

  8. [8]

    Least Squares Quantization in PCM,

    S. P. Lloyd, “Least Squares Quantization in PCM,”IEEE Transactions on Information Theory, vol. 28, no. 2, pp. 129–137, Mar. 1982

    work page 1982

Show all 27 references
  1. [9]

    8-bit Optimizers via Block-wise Quantization,

    T. Dettmers, M. Lewis, S. Shleifer, and L. Zettlemoyer, “8-bit Optimizers via Block-wise Quantization,” in Proc. of ICLR, virtual, Apr. 2022, pp. 1–19

    2022

  2. [10]

    Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs,

    J. Dotzel, Y . Chen, B. Kotb, et al., “Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs,” in Proc. of ICML, Vienna, Austria, Jul. 2024, pp. 11 573–11 591

    2024

  3. [11]

    Berger, Rate-Distortion Theory

    T. Berger, Rate-Distortion Theory. Wiley, 2003

    2003

  4. [12]

    The Llama 3 Herd of Models,

    A. Dubey, A. Jauhri, A. Pandey, et al., “The Llama 3 Herd of Models,” arXiv, Jul. 2024. arXiv: 2407.21783

    2024 arXiv

  5. [13]

    Qwen2.5 Technical Report,

    A. Yang, B. Yang, B. Zhang, et al., “Qwen2.5 Technical Report,” arXiv, Dec. 2024. arXiv: 2412.15115

    2024 arXiv

  6. [14]

    Mistral 7B,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, et al., “Mistral 7B,” arXiv, Oct. 2023. arXiv: 2310. 06825

    2023

  7. [15]

    Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor,

    O. Honovich, T. Scialom, O. Levy, and T. Schick, “Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor,” in Proc. of ACL, Toronto, ON, Canada, Jul. 2023, pp. 14 409–14 428

    2023

  8. [16]

    Magicoder: Empowering Code Generation with OSS-INSTRUCT,

    Y . Wei, Z. Wang, J. Liu, Y . Ding, and L. Zhang, “Magicoder: Empowering Code Generation with OSS-INSTRUCT,” in Proc. of ICML, Vienna, Austria, Jul. 2024, pp. 52 632–52 657

    2024

  9. [17]

    Pointer Sentinel Mixture Models,

    S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer Sentinel Mixture Models,” in Proc. of ICLR, Toulon, France, Apr. 2017, pp. 1–15

    2017

  10. [18]

    The LAMBADA Dataset: Word Prediction Requiring a Broad Discourse Context,

    D. Paperno, G. Kruszewski, A. Lazaridou, et al., “The LAMBADA Dataset: Word Prediction Requiring a Broad Discourse Context,” inProc. of ACL, Berlin, Germany, Aug. 2016, pp. 1525– 1534

    2016

  11. [19]

    Measuring Massive Multitask Language Under- standing,

    D. Hendrycks, C. Burns, S. Basart, et al., “Measuring Massive Multitask Language Under- standing,” in Proc. of ICLR, virtual, May 2021, pp. 1–27

    2021

  12. [20]

    Think You Have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge,

    P. Clark, I. Cowhey, O. Etzioni, et al., “Think You Have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge,” arXiv, Mar. 2018. arXiv: 1803.05457

    2018 arXiv

  13. [21]

    HellaSwag: Can a Machine Really Finish Your Sentence?

    R. Zellers, A. Holtzman, Y . Bisk, A. Farhadi, and Y . Choi, “HellaSwag: Can a Machine Really Finish Your Sentence?” In Proc. of ACL, Florence, Italy, Jul. 2019, pp. 4791–4800. 10

    2019

  14. [22]

    PIQA: Reasoning about Physical Commonsense in Natural Language,

    Y . Bisk, R. Zellers, R. L. Bras, J. Gao, and Y . Choi, “PIQA: Reasoning about Physical Commonsense in Natural Language,” in Proc. of AIII , New York, NY , USA, Feb. 2020, pp. 7432–7439

    2020

  15. [23]

    Social IQa: Commonsense Reasoning about Social Interactions,

    M. Sap, H. Rashkin, D. Chen, R. Le Bras, and Y . Choi, “Social IQa: Commonsense Reasoning about Social Interactions,” in Proc. of EMNLP-IJCNLP , Hong Kong, China, Nov. 2019, pp. 4463–4473

    2019

  16. [24]

    WinoGrande: An Adversarial Wino- grad Schema Challenge at Scale,

    K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y . Choi, “WinoGrande: An Adversarial Wino- grad Schema Challenge at Scale,” Commun. ACM, vol. 64, no. 9, 99–106, Aug. 2021

    2021

  17. [25]

    A table of normal integrals,

    D. B. Owen, “A table of normal integrals,” Communications in Statistics - Simulation and Computation, vol. 9, no. 4, pp. 389–419, 1980

    1980

  18. [26]

    R. L. Burden and J. D. Faires, Numerical Analysis. Brooks/Cole, 2010

    2010

  19. [27]

    Decoupled Weight Decay Regularization,

    I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” inProc. of ICLR, New Orleans, LA, USA, May 2019, pp. 1–10. 11 Appendix A Ablation on Constrained (i.e., Fixed) Reconstruction Levels In Tab. 5, we evaluate the importance of precisely representing zero weigh...

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.