REVIEW 5 cited by
Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements have demonstrated that the performance of large language models (LLMs) can be significantly enhanced by scaling computational resources at test time. A common strategy involves generating multiple Chain-of-Thought (CoT) trajectories and aggregating their outputs through various selection mechanisms. This raises a fundamental question: can models with lower complexity leverage their superior generation throughput to outperform similarly sized Transformers for a fixed computational budget? To address this question and overcome the lack of strong subquadratic reasoners, we distill pure and hybrid Mamba models from pretrained Transformers. Trained on only 8 billion tokens, our distilled models show strong performance and scaling on mathematical reasoning datasets while being much faster at inference for large batches and long sequences. Despite the zero-shot performance hit due to distillation, both pure and hybrid Mamba models can scale their coverage and accuracy performance past their Transformer teacher models under fixed time budgets, opening a new direction for scaling inference compute.
Forward citations
Cited by 5 Pith papers
-
Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
Reasoning-enabled LLMs show more robust performance on Theory of Mind tests under prompt and task perturbations, supporting a robustness-based reading of recent gains.
-
Raven: High-Recall Sequence Modeling with Sparse Memory Routing
Raven is a linear-time sequence model whose sparse, input-dependent routing writes tokens into dedicated memory slots, preserving long-context recall and extrapolating 16x beyond training length.
-
Kinetics: Rethinking Test-Time Scaling Laws
A memory-aware test-time scaling law shows small models are overestimated and sparse attention is needed for efficient scaling.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
-
Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Models
A probabilistic saturation model for test-time scaling is proposed and fitted to reasoning benchmarks, but the plateau 'prediction' is computed from the same per-problem data used to measure it.
Discussion (0). Sign in to comment.