Pith. sign in

REVIEW 5 cited by

Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.11860 v2 pith:CRZUFWNJ submitted 2023-05-19 cs.CL

Pranjal Aggarwal , Aman Madaan , Yiming Yang , Mausam This is my paper · ORCID
classification cs.CL
keywords adaptive-consistencyllmssamplesapproachavailablebudgetcodenumber
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency - poll the LLM multiple times and output the most frequent solution. Existing Self-Consistency techniques always generate a constant number of samples per question, where a better approach will be to non-uniformly distribute the available budget based on the amount of agreement in the samples generated so far. In response, we introduce Adaptive-Consistency, a cost-efficient, model-agnostic technique that dynamically adjusts the number of samples per question using a lightweight stopping criterion. Our experiments over 17 reasoning and code generation datasets and three LLMs demonstrate that Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%. Our code and data are available at https://www.sample-step-by-step.info

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VecFontLLM: Anchor-Guided Direct Synthesis of Chinese Vector Fonts

    cs.CV 2026-07 conditional novelty 7.0 of 10

    An anchor-guided MLLM generates complex Chinese vector glyphs from one or a few style exemplars, decoupling coarse layout from Bézier curve completion to improve structure and editability.

  2. TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

    cs.LG 2026-07 conditional novelty 6.5 of 10

    A heterogeneous-graph router jointly selects the optimal modality (text, vision, or both) and model per time series query, beating prior routing baselines and generalizing to unseen models and tasks.

  3. Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    DDC reduces token consumption by over 10x in LLM reasoning while maintaining or exceeding baseline accuracy across five benchmarks via adaptive path quality filtering.

  4. Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answers

    stat.ML 2026-02 conditional novelty 6.0 of 10

    Tracking only the counts of the two most frequent LLM answers is asymptotically as sample-efficient as the full Bayesian posterior in an adaptive self-consistency procedure, cutting LLM calls by up to half.

  5. EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Keeping only the first and last parts of chain-of-thought training traces preserves math reasoning accuracy on MATH500 at half the sequence length, but not on harder benchmarks.

Pith tools