Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithms

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that LLM decoding algorithms are consistent only when matched to their end goal: deterministic search for information retrieval, random sampling for creative generation.

desk verdict Interesting framing and one correct positive result, but the central impossibility theorem is false as stated, so the headline claim does not hold up. read the letter →

arxiv 2505.11183 v1 pith:EQ3TO7TJ submitted 2025-05-16 stat.ML cs.CLcs.LG

Jacob Trauger , Ambuj Tewari This is my paper · ORCID
classification stat.MLcs.CLcs.LG MSC 68Q3268T50
keywords next-tokenpredictiondecodingalgorithmslargelanguagemodelssurrogatelossconsistencyN-gramHammingcross-entropyinformationretrievalcreativegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether standard LLM decoding algorithms—greedy, lookahead, random sampling, and temperature-scaled sampling—are consistent for the two main ways people use language models: retrieving a correct answer and generating fresh samples from a distribution. It argues that these goals impose opposite requirements on decoders: deterministic algorithms are the only ones that can be optimal for a retrieval-style N-gram Hamming loss, while random sampling is necessary and sufficient to reproduce the true sequence distribution under cross-entropy loss. The paper also claims that optimal decoding for the retrieval objective is computationally hard in the worst case, requiring exponential time in sequence length for at least one next-token distribution. The practical upshot is that decoder choice should track user intent, and many common decoders lack theoretical grounding outside narrow classes of distributions.

What carries the argument

The central objects are the reformulated expected risk R(D, p, p_ntp, ell), which averages the loss over inputs, true outputs, and the decoder's randomized outputs, and the N-gram Hamming loss defined as the sum of indicators that each length-N block of the predicted sequence differs from the true block. The load-bearing identity is Lemma 4.1: minimizing this expected loss is equivalent to maximizing g(y) = sum_{i=1}^{L-N+1} p(y_{i:i+N-1}), the sum of N-gram marginals, which generalizes the known optimal-output characterizations for Hamming and 0-1 loss. This identity turns decoding into a combinatorial maximization problem on a tree of conditional probabilities, and it is the engine behind both the optimality characterization of K-T lookahead and the exponential lower bound. The dichotomy between retrieval and generation is carried by the difference between point-mass output distributions (deterministic decoders) and distributions that replicate the true sequence distribution (random sampling): the former can win on the N-gram objective but have infinite cross-entropy to non-deterministic targets, while the latter match the cross-entropy optimum but necessarily put mass on suboptimal sequences for retrieval.

What would settle it

Under the distribution p(y_i | y_{<i}) = 1/|V|, calculate g(y) for any two sequences: both equal (L-N+1) * |V|^{-N}. Because g is constant, the sequence 'aaaa...a' is an optimal output and can be produced without any next-token queries, so the claimed runtime lower bound of $\Omega$(|V|^{L-1}) fails for this distribution.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a formal inconsistency dichotomy. Fix a true next-token distribution p* and assume the trained predictor converges to it in KL divergence. For the N-gram Hamming loss—a sum over whether each length-N block of the decoded sequence matches the true block—the optimal output is the sequence maximizing the sum of N-gram marginal probabilities (Lemma 4.1). Deterministic decoders such as K-T lookahead can be consistent for this loss, but only on a restricted class of distributions characterized by a greedy-optimality condition (Theorem 4.4); and no deterministic decoder can be consistent everywhere, because the paper exhibits a distribution for which any optimal deterministic or stochastic decoder must query $\Omega$(|V|^{L-1}) next-token probabilities, yielding an exponential-time lower bound (Theorem 4.2, Corollary 4.2.1). For the cross-entropy loss on whole sequences, the situation flips: deterministic decoders incur infinite expected loss whenever the true sequence distribution is non-deterministic, while random sampling is consistent for every distribution (Proposition 5), and temperature scaling with parameter gamma != 1 recovers the true distribution only in uniform or deterministic cases (Proposition 6). Taken together, the paper claims that retrieval and generation are not served by the same decoding strategy.

Load-bearing premise

The hardness theorem assumes that a decoder facing a perfectly uniform next-token distribution must still search for a high-probability continuation, even though in that distribution every continuation is equally probable and every output is optimal.

Editorial extensions

If this is right

  • If the next-token predictor converges to the true distribution, then random sampling is consistent for the goal of generating samples from the true sequence distribution under cross-entropy loss, for every input distribution.
  • For information retrieval, there is no polynomial-time decoding algorithm that is optimal for all probability distributions; optimal N-gram Hamming decoding requires exponential time in sequence length in the worst case.
  • K-T lookahead, including greedy and beam search variants, is consistent for N-gram Hamming loss exactly on the set of distributions where the greedy path coincides with the global maximizer of the N-gram marginal sum; outside that set it can be suboptimal.
  • Increasing lookahead depth K or decreasing the number of committed tokens T does not, in general, enlarge the class of distributions for which K-T lookahead is consistent.
  • Temperature-scaled random sampling with gamma != 1 is consistent with the true sampling distribution only for uniform or deterministic distributions, and its suboptimality gap in cross-entropy grows at a rate linear in gamma.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The consistency argument for random sampling does not actually depend on the specific sampling order; any method that produces a sequence with probability equal to the product of the true conditionals will be cross-entropy consistent, so the result extends to other ancestral sampling schemes.
  • The N-gram Hamming framework can be applied to other N-gram-based evaluation metrics that are not simple sums of indicators, such as BLEU or ROUGE, by replacing the loss with a smoothed or clipped variant; the optimal-output characterization would need to be re-derived for those non-additive losses.
  • The exponential lower bound is stated for a fully uniform conditional distribution; a natural testable extension is whether the bound survives arbitrarily small perturbations away from uniform, since in the exact uniform case any fixed output is already optimal.
  • The dichotomy suggests a testable design principle for adaptive decoding: algorithms that shift continuously between deterministic and stochastic behavior, such as temperature adjustment, will be consistent for at most one of the two objectives unless the true distribution is degenerate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies decoding of next-token predictors under two target objectives: the N-gram Hamming loss, intended to model information retrieval, and the cross-entropy loss on full output sequences, intended to model creative generation. It analyzes K-T lookahead (including greedy), random sampling, and temperature-scaled random sampling, under the assumption that the next-token predictor converges to the true conditional distribution. The paper claims (i) random sampling is consistent for the cross-entropy objective, (ii) deterministic decoders have infinite cross-entropy loss for non-deterministic targets, (iii) no polynomial-time decoder is optimal for all probability distributions for the N-gram Hamming loss, and (iv) K-T lookahead is optimal only on a restricted class of distributions. The appendix contains proofs and a simulation study on Markov chains.

Significance. The surrogate-loss-consistency framing for LLM decoding is a reasonable and potentially useful contribution, and the positive result that random sampling from converged next-token probabilities matches the target sequence distribution in cross-entropy is correct and cleanly proved in Appendix B.13. However, the manuscript's headline impossibility result is false as stated, and the main characterization of K-T lookahead is a tautology. Since these items support the abstract's central dichotomy and the claim that the studied decoders lack theoretical grounding outside a subset of distributions, the paper does not currently establish its main claims. If the negative results were replaced by a valid distribution-free adversary argument and a non-vacuous characterization, the framework could be valuable, but the present version is not publishable in its current form.

major comments (3)
  1. [§4.2, Theorem 4.2 and Appendix B.5] Theorem 4.2 is false as stated. Under the hypothesis p(y_i | y[i-1]) = 1/|V| for every prefix, every sequence has probability |V|^{-L}, every N-gram marginal equals |V|^{-N}, and the function g(y) in Lemma 4.1 is constant on Y. Consequently every output is optimal, and a decoder that always outputs a fixed sequence is optimal in O(1) time, contradicting the claimed Ω(|V|^{L-1}) lower bound. The key step in Appendix B.5, 'If either of these nodes have a weight more than 1/|V|', is never satisfied under the stated distribution. The surrounding discussion suggests the intended claim is a distribution-free lower bound, but that requires constructing an alternate distribution that agrees with the uniform conditional on every queried (prefix, token) pair and has a different unique N-gram-Hamming optimum. The proof neither states nor constructs such an adversary. Corollary 4.2.1 therefore does not follow.
  2. [§4.3, Theorem 4.4] The characterization is vacuous. The set C is defined as the set of inputs x for which the sequence produced by repeatedly taking conditional arg maxs (the K-T lookahead output) equals the global arg max of g(y); the theorem then asserts that K-T lookahead is optimal iff p*_x(C)=1. This is a restatement of the definition of optimality rather than a characterization of the distributions for which the decoder is consistent. It provides no structural condition on p* that could be checked or falsified, and it does not support the conclusions drawn in the surrounding discussion or in the simulation analysis. The notation (arg max ...)_{[T]} also needs to be defined explicitly.
  3. [§5.2, Proposition 5 vs. the consistency definition in §3] There is a mismatch between the consistency definition and the cross-entropy claim. Section 3 defines consistency as convergence to inf_{h:X→Y} R(h, p*, ℓ), where h is a deterministic decoder. For a non-deterministic p*, the cross-entropy risk of any deterministic h is infinite, so the infimum is +∞, while the proof of Proposition 5 shows CE(p*, p_i) → H(p*) < ∞. Under the stated definition, random sampling would not be consistent; the proof implicitly changes the benchmark to the minimum over all output distributions. Either the consistency definition must be extended to randomized decoders or to distribution-valued outputs, or Proposition 5 must be restated as distribution matching rather than consistency in the earlier sense.
minor comments (4)
  1. [§4.2, Theorem 4.2 statement] The theorem says '∀i∈L'; the index set should be [L] (or {1,...,L-1} for the prefix conditionals).
  2. [§4.4.1] The sentence 'by Proposition 1 and Theorem 4.2, neither of these are consistent for all probability distributions' is misleading: Theorem 4.2 is a lower bound for optimal decoders and does not directly apply to random or temperature-scaled sampling. The inconsistency follows from Proposition 4; please correct the reference.
  3. [Appendix B.15, proof of Proposition 7] The sign conventions for the constants are inconsistent: the lower bound in (1) is written as γC1 - opt, but the proof's inequality (⋆) ≥ -γ C_{2,i} + log|V|, when summed, gives a γ coefficient of -Σ C_{2,i}, which is negative, so C1 cannot be a positive integer with the stated inequality unless the constants are redefined. Please define C1, C2, C3 explicitly and verify the displayed bounds.
  4. [Figures 1-6] The figures would be easier to read with explicit axis labels and with the tie-breaking rule stated in the captions; Appendix A.3 explains the tie handling only in the text.

Circularity Check

1 steps flagged · score 4.0 of 10

One self-definitional characterization (Theorem 4.4); other results are not circular, while Theorem 4.2 has a serious correctness gap that is not itself a circularity.

  1. self definitional [Section 4.3, Theorem 4.4; proof in Appendix B.7]
    "By the defintion of C, we know the KT -lookahead outputs maximize P L−N+1 i=1 p∗(yi:i+N−1|x) except a set of measure0 over X. From Lemma 4.1, we can see that this is the optimal output. ... Then, KT -lookahead is N-gram Hamming loss optimal for p∗ iff p∗ x(C) = 1."

    The set C is defined as exactly the set of inputs x on which the KT-lookahead trajectory y† coincides with the global N-gram-Hamming arg max over Y. The forward implication is asserted directly from this definition, and the converse assumes the negation of the defining condition (y† is the global arg max while the decoder outputs ŷ ≠ y†). Thus Theorem 4.4 restates the target optimality property by construction rather than deriving an independent characterization. Corollary 4.4.1 repeats the same definitional move with arg max_B in place of arg max.

full rationale

The only step that reduces to its own definition is Theorem 4.4 (and Corollary 4.4.1), where the condition p∗_x(C)=1 is defined as the event that the lookahead output already equals the global N-gram-Hamming arg max; the theorem is therefore a restatement of optimality rather than an independent condition. This is a partial, localized circularity and is not load-bearing for the paper's main independent results: random-sampling consistency under cross entropy (Proposition 5), the temperature-scaling suboptimality characterization (Proposition 6), the rate bounds (Proposition 7), and the explicit counterexample distributions in Propositions 2 and 3 are all derived from the stated assumptions without smuggling in their conclusions. I find no load-bearing self-citation chain and no ansatz-smuggled-in-via-citation pattern. One important note on validity rather than circularity: Theorem 4.2's proof appears flawed as written, because under the stated distribution p(y_i|y[i-1])=1/|V| every node has weight exactly 1/|V|, so the adversarial node with weight 'more than 1/|V|' cannot exist and g(y) is constant; a correct distribution-free lower bound would require constructing an alternative distribution consistent with the algorithm's queries. That is a correctness gap, not a circular derivation, so it does not by itself raise the circularity score.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claims rest mainly on Assumption 3.1 and on the computational query model. The N-gram Hamming loss is a new loss function but not an invented entity. No new particles, forces, or model components are introduced.

free parameters (1)
  • temperature gamma = not fitted; user-chosen hyperparameter
    Temperature scaling parameter in the temperature-sampled decoder; Propositions 6 and 7 characterize consistency and rates as functions of gamma, so results are conditional on this choice.
assumptions (3)
  • domain assumption The next-token predictor converges to the true conditional distribution in KL divergence (Assumption 3.1).
    Stated in Section 3 and used in all consistency theorems; it is an idealized asymptotic assumption not guaranteed for real finite, misspecified LLMs.
  • ad hoc to paper The class P excludes distributions with ties in any arg max used by lookahead.
    Theorem 4.4 and Propositions 2 and 3 restrict to tie-free distributions; the paper notes the lower-bound example can be perturbed into P but does not prove the characterization outside P.
  • domain assumption Decoder runtime is measured in number of conditional probability queries, with each query costing C.
    Theorem 4.2's lower bound is stated in this black-box query model; real inference may batch or cache queries, so the complexity result is model-specific.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithms." pith.science (2026). https://pith.science/paper/EQ3TO7TJ

@misc{pith2026250511183,
  author       = {Pith},
  title        = {Pith review of: On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EQ3TO7TJ}},
  note         = {Machine review of arXiv:2505.11183}
}
read the original abstract

Probabilistic next-token prediction trained using cross-entropy loss is the basis of most large language models. Given a sequence of previous values, next-token prediction assigns a probability to each possible next value in the vocabulary. There are many ways to use next-token prediction to output token sequences. This paper examines a few of these algorithms (greedy, lookahead, random sampling, and temperature-scaled random sampling) and studies their consistency with respect to various goals encoded as loss functions. Although consistency of surrogate losses with respect to a target loss function is a well researched topic, we are the first to study it in the context of LLMs (to the best of our knowledge). We find that, so long as next-token prediction converges to its true probability distribution, random sampling is consistent with outputting sequences that mimic sampling from the true probability distribution. For the other goals, such as minimizing the 0-1 loss on the entire sequence, we show no polynomial-time algorithm is optimal for all probability distributions and all decoding algorithms studied are only optimal for a subset of probability distributions. When analyzing these results, we see that there is a dichotomy created between the goals of information retrieval and creative generation for the decoding algorithms. This shows that choosing the correct decoding algorithm based on the desired goal is extremely important and many of the ones used are lacking theoretical grounding in numerous scenarios.

Figures

Figures reproduced from arXiv: 2505.11183 by the authors.

Figure 1
Figure 1. A plot of the amount of trials K1-lookahead was optimal for the 1-gram Hamming loss (the Hamming loss). Each point represents the average optimality over 200 randomly generated Markov chains with a set amount of nodes and Dirichlet parameter α. Smaller αs create more “peaky” distributions and thus have higher KL-divergence from the uniform distribution, while larger αs create more uniform distributions. There were 8… view at source ↗
Figure 2
Figure 2. A plot of the amount of trials K1-lookahead was optimal for the L-gram Hamming loss (the 0 − 1 loss). The same setup as [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. A plot of all trials of K1-lookahead decoding being optimal for N-gram Hamming on a Markov chain with “num nodes” amount of nodes. The amount of nodes increases as one looks down the rows and the sequence length increases as one goes down the columns. Each line represents a specific K1-lookahead being compared to the optimal N-gram Hamming. The x and y axes denote the same that has been used in Figures 1 and 2. From… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: A plot of the amount of trials KK-lookahead was optimal for the 1-gram Hamming loss (the Hamming loss). There were 8 nodes in each Markov chain and the sequence length goes up by two as one moves right in the plots [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: A plot of the amount of trials KK-lookahead was optimal for the L-gram Hamming loss (the 0 − 1 loss). The same setup as [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: A plot of all trials of KK-lookahead decoding being optimal for N-gram Hamming on a Markov chain with “num nodes” amount of nodes. The amount of nodes increases as one looks down the rows and the sequence length increases as one goes down the columns. Each line represe…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Constraint-aware Learning of Probabilistic Sequential Models for Multi-Label Classification

    cs.LG 2025-07 conditional novelty 6.0 of 10

    An autoregressive 'sequence' model fed with per-label probability scores improves constrained multi-label classification on most tested datasets and learns logical constraints directly from the data.

Reference graph

Works this paper leans on

34 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    and Nagarajan, V

    Bachmann, G. and Nagarajan, V. (2024). The pitfalls of next-token prediction. arXiv preprint arXiv:2403.06963

    arXiv 2024

  3. [3]

    L., Jordan, M

    Bartlett, P. L., Jordan, M. I., and McAuliffe, J. D. (2006). Convexity, classification, and risk bounds. Journal of the American Statistical Association , 101(473):138–156

    work page 2006

  4. [4]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877--1901

    2020

  5. [5]

    W., Sutton, C., Gehrmann, S., et al

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2023). Palm: Scaling language modeling with pathways. Journal of Machine Learning Research , 24(240):1--113

    work page 2023

  6. [6]

    Dembczyński, K., Waegeman, W., Cheng, W., and Hüllermeier, E. (2010). Regret analysis for performance metrics in multi-label classification: The case of hamming and subset zero-one loss. In Balcázar, J. L., Bonchi, F., Gionis, A., and Sebag, M., editors, Machine Learning and Knowledge Discovery in Databases , page 280–295, Berlin, Heidelberg. Springer

    work page 2010

  7. [7]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , pages 4171--4186

    2019

  8. [8]

    Dhuliawala, S., Kulikov, I., Yu, P., Celikyilmaz, A., Weston, J., Sukhbaatar, S., and Lanchantin, J. (2024). Adaptive decoding via latent preference optimization. arXiv preprint arXiv:2411.09661

    arXiv 2024

Show all 34 references
  1. [9]

    Fu, Y., Bailis, P., Stoica, I., and Zhang, H. (2024). Break the sequential dependency of llm inference using lookahead decoding. arXiv preprint arXiv:2402.02057

    2024 arXiv

  2. [10]

    and Zhou, Z.-H

    Gao, W. and Zhou, Z.-H. (2011). On the consistency of multi-label learning. In Proceedings of the 24th Annual Conference on Learning Theory , page 341–358. JMLR Workshop and Conference Proceedings

    2011

  3. [11]

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. (2025). Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  4. [12]

    Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Carney, A., et al. (2024). Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  5. [13]

    O., Natarajan, N., Ravikumar, P

    Koyejo, O. O., Natarajan, N., Ravikumar, P. K., and Dhillon, I. S. (2015). Consistent multilabel classification. Advances in Neural Information Processing Systems , 28

    2015

  6. [14]

    K., Pitrou, A., and Seibert, S

    Lam, S. K., Pitrou, A., and Seibert, S. (2015). Numba: A llvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC , pages 1--6

    2015

  7. [15]

    E., Rawat, A

    Li, Y., Huang, Y., Ildiz, M. E., Rawat, A. S., and Oymak, S. (2024). Mechanics of next token prediction with self-attention. In International Conference on Artificial Intelligence and Statistics , pages 685--693. PMLR

    2024

  8. [16]

    Lin, C.-Y. (2004). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74--81

    2004

  9. [17]

    Lin, P., Zhang, Z., and Xu, Z.-Q. J. (2025). Reasoning bias of next token prediction training. arXiv preprint arXiv:2502.02007

    2025 arXiv

  10. [18]

    and Giesselbach, S

    Paa , G. and Giesselbach, S. (2023). Foundation models for natural language processing: Pre-trained language models integrating media . Springer Nature

    2023

  11. [19]

    Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311--318

    2002

  12. [20]

    H., and Riedel, S

    Petroni, F., Rockt \"a schel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. (2019). Language models as knowledge bases? arXiv preprint arXiv:1909.01066

    2019 arXiv

  13. [21]

    Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. (2018). Improving language understanding by generative pre-training

    2018

  14. [22]

    Ramaswamy, H. G. and Agarwal, S. (2016). Convex calibration dimension for multiclass loss matrices. Journal of Machine Learning Research , 17(14):1--45

    2016

  15. [23]

    G., Agarwal, S., and Tewari, A

    Ramaswamy, H. G., Agarwal, S., and Tewari, A. (2013). Convex calibrated surrogates for low-rank loss matrices with applications to subset ranking losses. Advances in Neural Information Processing Systems , 26

    2013

  16. [24]

    Saunshi, N., Malladi, S., and Arora, S. (2021). A mathematical exploration of why language models help solve downstream tasks. In International Conference on Learning Representations

    2021

  17. [25]

    and Ben-David, S

    Shalev-Shwartz, S. and Ben-David, S. (2014). Understanding machine learning: From theory to algorithms . Cambridge University Press

    2014

  18. [26]

    Shi, C., Yang, H., Cai, D., Zhang, Z., Wang, Y., Yang, Y., and Lam, W. (2024). A thorough examination of decoding methods in the era of llms. arXiv preprint arXiv:2402.06925

    2024 arXiv

  19. [27]

    Snell, C., Lee, J., Xu, K., and Kumar, A. (2024). Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

    2024 arXiv

  20. [28]

    and Bartlett, P

    Tewari, A. and Bartlett, P. L. (2007). On the consistency of multiclass classification methods. Journal of Machine Learning Research , 8(36):1007--1025

    2007

  21. [29]

    Thrampoulidis, C. (2024). Implicit optimization bias of next-token prediction in linear models. arXiv preprint arXiv:2402.18551

    2024 arXiv

  22. [30]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  23. [31]

    Wiher, G., Meister, C., and Cotterell, R. (2022). On decoding strategies for neural text generators. Transactions of the Association for Computational Linguistics , 10:997--1012

    2022

  24. [32]

    and Zhu, J

    Wu, G. and Zhu, J. (2020). Multi-label classification: do hamming loss and subset accuracy really conflict with each other? In Advances in Neural Information Processing Systems , volume 33, page 3130–3140. Curran Associates, Inc

    2020

  25. [33]

    Zhu, W., Hao, H., He, Z., Ai, Y., and Wang, R. (2024a). Improving open-ended text generation via adaptive decoding. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceedings of the 41st International Conference...

    2024

  26. [34]

    Zhu, Y., Li, J., Li, G., Zhao, Y., Jin, Z., and Mei, H. (2024b). Hot or cold? adaptive temperature sampling for code generation with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 437--445

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.