Pith. sign in

REVIEW 10 cited by

From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.16838 v2 pith:3WYKEHWN submitted 2024-06-24 cs.CL cs.LG

Sean Welleck , Amanda Bertsch , Matthew Finlayson , Hailey Schoelkopf , Alex Xie , Graham Neubig , Ilia Kulikov , Zaid Harchaoui This is my paper
classification cs.CLcs.LG
keywords algorithmsgenerationlanguagemeta-generationtoken-levelcomputedecodingduring
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

One of the most striking findings in modern research on large language models (LLMs) is that scaling up compute during training leads to better results. However, less attention has been given to the benefits of scaling compute during inference. This survey focuses on these inference-time approaches. We explore three areas under a unified mathematical formalism: token-level generation algorithms, meta-generation algorithms, and efficient generation. Token-level generation algorithms, often called decoding algorithms, operate by sampling a single token at a time or constructing a token-level search space and then selecting an output. These methods typically assume access to a language model's logits, next-token distributions, or probability scores. Meta-generation algorithms work on partial or full sequences, incorporating domain knowledge, enabling backtracking, and integrating external information. Efficient generation methods aim to reduce token costs and improve the speed of generation. Our survey unifies perspectives from three research communities: traditional natural language processing, modern LLMs, and machine learning systems.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. APPO: Agentic Procedural Policy Optimization

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    APPO refines branching and credit assignment in agentic RL via a Branching Score and procedure-level scaling, improving baselines by nearly 4 points on 13 benchmarks.

  2. More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

    cs.LG 2026-01 conditional novelty 6.0 of 10

    ReD, which resets after each attempt and discards solved problems, provably increases coverage@cost (unique solved problems per fixed budget) over solve-to-completion for LLMs with power-law pass@k, and can be used to...

  3. LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization

    cs.AI 2025-07 conditional novelty 6.0 of 10

    LAPO is a two-stage RL method that reduces reasoning token usage by up to 40.9% while slightly improving accuracy, by learning per-problem length targets from the model's own correct answers and prompting the model to...

  4. Breaking the Myth: Can Small Models Infer Postconditions Too?

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Fine-tuning Qwen2.5-Coder-7B on 1.5K reasoning examples yields postcondition generation close to GPT-4o and better than 32B open models on Defects4J.

  5. Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A fitted progress vector in LLM hidden states can monitor and, when added during decoding, shorten the thinking phase, often improving accuracy and cutting tokens.

  6. EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Keeping only the first and last parts of chain-of-thought training traces preserves math reasoning accuracy on MATH500 at half the sequence length, but not on harder benchmarks.

  7. Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning

    cs.AI 2025-06 conditional novelty 6.0 of 10

    Reasoning tokens like 'Hmm' and 'Wait' mark steps where a model's internal state carries unusually high dependence with the correct answer, and suppressing them hurts accuracy.

  8. Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A semantic entropy-guided stopping rule for multi-round parallel LLM reasoning improves accuracy while reducing inference steps on five benchmarks.

  9. Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    ADAPT, a diversity-aware prefix fine-tuning method, improves best-of-N sampling efficiency for a 1.5B reasoning model, reaching 80% accuracy at N=32 versus N=256 for the baseline.

  10. First Finish Search: Efficient Test-Time Scaling in Large Language Models

    cs.CL 2025-05 conditional novelty 4.0 of 10

    First Finish Search launches n parallel reasoning traces and returns the shortest one, improving accuracy on AIME benchmarks while cutting token use.

Pith tools