Pith. sign in

REVIEW 9 cited by

Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.03094 v3 pith:VG2JB46E submitted 2023-10-04 cs.CL cs.AIcs.LG

Murong Yue , Jie Zhao , Min Zhang , Liang Du , Ziyu Yao This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords llmsperformancereasoningstrongerweakeranswercascadecascades
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) such as GPT-4 have exhibited remarkable performance in a variety of tasks, but this strong performance often comes with the high expense of using paid API services. In this paper, we are motivated to study building an LLM cascade to save the cost of using LLMs, particularly for performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline follows the intuition that simpler questions can be addressed by a weaker but more affordable LLM, whereas only the challenging questions necessitate the stronger and more expensive LLM. To realize this decision-making, we consider the "answer consistency" of the weaker LLM as a signal of the question difficulty and propose several methods for the answer sampling and consistency checking, including one leveraging a mixture of two thought representations (i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can achieve performance comparable to using solely the stronger LLM but require only 40% of its cost.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Using conformal prediction-set size as the cascade deferral rule yields distribution-free cascade accuracy bounds and usually beats confidence-threshold heuristics on multiple-choice LLM benchmarks.

  2. HACO: Hedged Agent Computing for Reliable LLM Systems

    cs.NI 2026-07 conditional novelty 6.0 of 10

    HACO adaptively selects a reliability-constrained hedge set of candidate LLM agent instances for each role invocation, improving robustness under changing deployment conditions at lower token/latency cost than full-po...

  3. BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A routing system that chooses both the model and the number of samples per query to meet a quality threshold, yielding up to 60% cost savings.

  4. AdaptiveLLM: A Framework for Selecting Optimal Cost-Efficient LLM for Code-Generation Based on CoT Length

    cs.SE 2025-06 conditional novelty 6.0 of 10

    AdaptiveLLM routes coding problems to cost-effective LLMs by clustering chain-of-thought lengths into three difficulty levels, improving pass@1 by 7.86% over ComplexityNet while cutting inference cost by 88.9%.

  5. When One LLM Drools, Multi-LLM Collaboration Rules

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A position paper that introduces a four-level taxonomy of multi-LLM collaboration (API, text, logit, weight) and argues it is essential for reliability, pluralism, and democratization.

  6. Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

    stat.ML 2026-07 conditional novelty 5.5 of 10

    Jointly calibrated early thought-stopping and perplexity deferral cut edge-agent thinking compute 43–73% while keeping finite-sample reward and cloud-call guarantees.

  7. How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Value-weighted LLM routing matches difficulty-only recall while raising precision, exposes within-category calibration collapse, and an elastic value-scaled budget absorbs a synthetic Black Friday surge.

  8. Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes

    cs.CL 2026-08 conditional novelty 4.0 of 10

    Statistical classifiers built on LLM activation norms and coordinates match or beat trained MLP heads on coarse intent routing and resist camouflage better, while MLPs win on fine-grained subfield distinctions.

  9. Token-Operations-Oriented Inference Optimization Techniques for Large Models

    cs.SE 2026-06 unverdicted novelty 3.0 of 10

    The paper introduces a four-layer technical architecture for token-operations-oriented inference optimization in large models and reviews key technologies and industry status at each layer.

Pith tools