REVIEW 9 cited by
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language model (LLM) scaling laws are empirical formulas that estimate changes in model quality as a result of increasing parameter count and training data. However, these formulas, including the popular Deepmind Chinchilla scaling laws, neglect to include the cost of inference. We modify the Chinchilla scaling laws to calculate the optimal LLM parameter count and pre-training data size to train and deploy a model of a given quality and inference demand. We conduct our analysis both in terms of a compute budget and real-world costs and find that LLM researchers expecting reasonably large inference demand (~1B requests) should train models smaller and longer than Chinchilla-optimal. Furthermore, we train 47 models of varying sizes and parameter counts to validate our formula and find that model quality continues to improve as we scale tokens per parameter to extreme ranges (up to 10,000). Finally, we ablate the procedure used to fit the Chinchilla scaling law coefficients and find that developing scaling laws only from data collected at typical token/parameter ratios overestimates the impact of additional tokens at these extreme ranges.
Forward citations
Cited by 9 Pith papers
-
Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems
MAAC defines a nine-dimension framework for process-oriented cognitive evaluation of text-based AI, shifting assessment from outputs to underlying reasoning.
-
MIRROR: Learning from the Other View for Multi-Modal Reasoning
An RL method that selects the best-performing view of each geometry problem as an internal teacher and distills it into weaker views improves VLM reasoning accuracy and consistency.
-
What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs
VLM accuracy can be predicted from a scalar capability score derived from LLM text benchmarks plus multimodal data volume via a fitted transfer-absorption scaling law.
-
Towards Distributed Neural Architectures
A general routing framework lets each token take its own path through a network, matching dense baselines while enabling learned compute savings and interpretable path specialization.
-
A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search
A skill-graph random-walk model gives closed-form accuracy-versus-compute formulas for four reasoning strategies and connects them to training scaling.
-
AbbIE: Autoregressive Block-Based Iterative Encoder for Efficient Sequence Modeling
An iterative Transformer that reuses its middle layers with an extra residual connection matches a standard Transformer after one pass and improves perplexity and zero-shot accuracy when run for more passes at test time.
-
Spectra 1.1: Scaling Laws and Efficient Inference for Ternary Language Models
Ternary language models trained on 1.2 trillion tokens continue to improve, and a new GPU kernel speeds up their inference up to 5x end-to-end.
-
Agent-based Condition Monitoring Assistance with Multimodal Industrial Database Retrieval Augmented Generation
MindRAG retrieves similar historical vibration recordings and maintenance annotations, then uses LLM agents to generate fault predictions and alarm recommendations for industrial condition monitoring.
-
The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy
Medical agents should be scaled mainly by richer clinical environments and self-evolution loops, not parameter growth alone, under a three-level autonomy taxonomy.
Discussion (0). Continue with ORCID to comment.