Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Mixture-of-experts models deliver the best accuracy per inference FLOP on math and science reasoning benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:14 UTC pith:DHNEOZB3

load-bearing objection Useful snapshot of compute-accuracy trade-offs across 19 open models, but the MoE 'architectural advantage' is confounded by model vintage/tuning and an unvalidated FLOPs proxy, so treat the headline as suggestive. the 4 major comments →

arxiv 2512.24776 v1 pith:DHNEOZB3 submitted 2025-12-31 cs.CL

Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Models

\'Akos Prucs , Nara Csutora , M\'aty\'as Antal , M\'ark Marosi This is my paper
classification cs.CL
keywords Pareto frontiermixture of expertsinference computechain-of-thoughttest-time scalingFLOPs estimationreasoning benchmarkstrace length asymmetry
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the right way to compare reasoning LLMs is by accuracy per unit of inference compute, not raw benchmark score. Using a component-aware FLOPs estimator and a single-pass chain-of-thought protocol on five math and science benchmarks, it argues that mixture-of-experts architectures convert FLOPs into accuracy more efficiently than dense models, that test-time scaling saturates at a task-dependent knee, and that failed reasoning traces systematically burn more compute than successful ones. A sympathetic reader would care because the result gives a concrete selection rule for compute-bounded deployments and points to sparsity as a structural enabler for reasoning.

Core claim

The paper reports that when open-source reasoning models are compared by accuracy against estimated FLOPs per answer under a single-pass chain-of-thought evaluation across five math and science benchmarks, mixture-of-experts models sit on the Pareto frontier more often than dense models. It also finds that incorrect reasoning traces are systematically longer and more compute-intensive than correct ones—97% of evaluated models show this asymmetry—and that each benchmark has a task-dependent compute knee beyond which added FLOPs yield negligible accuracy gains. The authors present these as empirical regularities: sparsity enables longer reasoning chains within a fixed FLOP budget, extended rea

What carries the argument

The central object is the component-aware FLOPs estimator (Equations 1–7), which decomposes transformer inference cost per token into attention projections adjusted for grouped-query attention, gated versus standard feed-forward networks, mixture-of-experts active experts plus a router term, quadratic attention as a function of sequence length, Mamba layer projections, and the vocabulary head. This estimator feeds the construction of the empirical Pareto frontier and the scalar Efficiency Score, which measures average accuracy per order of magnitude of compute. The estimator is what makes the MoE architectural advantage visible: because only a subset of experts is active per token, keeping t

Load-bearing premise

The paper counts compute using a formula that assumes an MoE model's cost is just the active experts' work plus a small router term, ignoring the real-world overheads of moving data between experts and memory; if that assumption is wrong, MoE's top ranking on the frontier could disappear.

What would settle it

Measure actual end-to-end serving cost—wall-clock time per query under realistic batch load, or energy per completed answer—for the best MoE and dense models on the same five benchmarks. If a dense model matches or beats the MoE model on accuracy per real-world dollar or watt, the paper's Equation (4) proxy has inverted the ranking.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Under the paper's FLOPs accounting, mixture-of-experts becomes the natural choice for compute-bounded reasoning deployments.
  • Every benchmark has a compute knee; beyond it, buying more tokens buys negligible accuracy, so inference budgets can be capped per task without expected accuracy loss.
  • Because incorrect answers are systematically more expensive than correct ones, average compute per query overstates the cost of successful reasoning; early stopping and uncertainty-aware termination are promising levers.
  • Smaller models can reach Pareto-optimal points by writing much longer reasoning traces, making model size and inference-time compute substitutes rather than independent axes.
  • The efficiency score's upward trajectory over release dates indicates that newer open reasoning models produce more accuracy per order of magnitude of FLOPs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct consequence the paper leaves implicit: if MoE's FLOPs advantage is real, its edge should widen as reasoning traces lengthen, because each extra token is cheaper in a sparse model; a matched comparison of accuracy-versus-trace-length curves would test this.
  • The paper evaluates single completions only; whether MoE Pareto dominance persists under majority voting or verifier reranking, where dense models can spend compute on multiple samples, is an open question.
  • Because the estimator ignores memory bandwidth, inter-expert communication, and other serving overheads, the real-world frontier measured in dollars or watts may differ; the MoE claim should be read as about FLOPs, not about end-to-end serving cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript evaluates 19 open-weight LLMs on five math/science reasoning benchmarks (GSM8K, MATH500, AIME25, HMMT-Feb-25, GPQA-Diamond) under a single-pass, full-Chain-of-Thought protocol. It estimates per-query inference FLOPs from architectural parameters and generation lengths (Eqs. 1–7), plots accuracy against FLOPs, and constructs Pareto frontiers. The central claims are: (i) MoE architectures dominate the dense models on the Pareto frontier; (ii) test-time compute exhibits task-dependent saturation knees; (iii) a temporal efficiency score improves from 2023 to 2025; (iv) wrong reasoning traces consume substantially more compute than correct ones. The paper also compares reproduced accuracies with official published numbers in Table 2.

Significance. If the MoE result were robust, the paper would make a useful practical contribution to compute-aware model selection and would provide evidence that sparse architectures convert inference FLOPs into reasoning accuracy more efficiently. The manuscript is transparent about architecture parameters, uses a consistent evaluation protocol, and reports a side-by-side comparison with published accuracies, which is a strength. However, the headline MoE conclusion is currently under-supported: the model pool confounds architecture with release date and reasoning-specific tuning, and the FLOPs estimator is an unvalidated theoretical construction that may already presume the MoE advantage. The accuracy discrepancies in Table 2 further undermine the reliability of the frontiers. With additional matched comparisons, metric validation, and uncertainty quantification, this could become a valuable study; in its present form the central claims are not established.

major comments (4)
  1. [§4.3, Figure 2] The claim that the evaluation 'isolates a distinct structural advantage' for MoE is not supported by the model pool. All MoE entries are 2025 reasoning-tuned models (Qwen3-30B-A3B Thinking/Instruct, gpt-oss-20b), while dense entries include older general instruct models (Llama-3.1-8B-Instruct, Gemma-3 base, phi-4). The only dense reasoning-tuned model at a comparable scale, Qwen3-4B-Thinking, is much smaller in total parameters, and Phi-4-reasoning-plus is not matched in training recipe. The MoE dominance in Figure 2 could therefore reflect release date, training data, or RL/FT regime rather than architecture. Provide matched dense/MoE pairs (e.g., same family and release), or a covariate-adjusted comparison; the Limitations section (4.4) only tempers the temporal trend, not the MoE claim.
  2. [Eq. (4), Eq. (7), §4.3] The MoE efficiency result is partly built into the metric. Eq. (4) counts only the k active experts plus a small router term, and Eq. (7) uses this to define per-query FLOPs. Under this definition, MoE necessarily 'decouples generation length from total FLOP consumption' (Sec. 4.3). In real serving, memory bandwidth to load all experts, all-to-all communication, and other overheads are substantial; active-parameter FLOPs may not reflect latency, energy, or cost. The paper provides no external validation of the FLOPs counts against measured runtime or throughput. Without such validation, the x-axis of Figure 2 is a modeling artifact rather than an empirical cost, and the MoE frontier position is a consequence of the estimator.
  3. [Table 2, §3.3] Reproduced accuracies diverge sharply from official numbers for several models: DeepSeek-R1-0528-Qwen3-8B AIME25 76.3->50.0 and HMMT 61.5->23.3; phi-4 AIME25 62.9->20.0; Phi-4-reasoning-plus GPQA-Diamond 68.9->33.8. The text attributes such gaps in general terms to decoding parameters, prompt formats, or majority-vote aggregation, but no analysis is provided. Since the Pareto frontiers and MoE conclusions are built on the reproduced accuracies, these unexplained deviations could reorder the frontiers. Report per-item errors, binomial confidence intervals, and multiple seeds (AIME25 has only 30 items), and demonstrate that the deviations are not correlated with architecture.
  4. [§4.1, §4.4, Eq. (8)] The 'deductive horizon' and the 'Pareto knee' are central to claims (iv) and (vi), but neither is defined operationally; no statistical procedure is used to locate knees. The temporal efficiency score S_eff is an ad hoc metric: mean accuracy divided by log10(total FLOPs). This normalization makes the temporal trend in Figure 3 depend entirely on the chosen denominator; a model with moderately lower accuracy but much lower compute can score higher. No sensitivity analysis or alternative normalization is reported.
minor comments (5)
  1. [§4.2] Typo: 'This empirically validates the that "thinking compute" acts as...' should read 'validates that'.
  2. [Table 2] The HMMT cell for gpt-oss-20b-reason-low reads '– / 2.'; this appears truncated. Also, Phi-4-mini-flash-reasoning has dashes for all reproduced values though it is listed in the model pool; please clarify whether it was evaluated.
  3. [§3.2] A universal frequency penalty of 0.05 is introduced without comparison to the default (0.0). Since this differs from the 'official' configurations used to produce the published accuracies in Table 2, its effect on the reproduced scores should be quantified or at least justified.
  4. [§3.3] The judge is given only the final 20 tokens of the model output. This is a very short window; models that do not strictly follow the 'Answer:' or '\boxed{}' format may have their correct answer placed earlier. Report the agreement between the regex parser and the judge, and the failure rate of the regex pass.
  5. [References] Several references have placeholder-style arXiv identifiers (e.g., arXiv:2412.00000, arXiv:2504.00000, arXiv:2503.00000, arXiv:2505.00000). These need to be replaced with correct citations before publication.

Circularity Check

1 steps flagged

MoE efficiency advantage is partially definitional: Eq. (4)'s active-expert FLOPs model is restated as the Section 4.3 architectural conclusion.

specific steps
  1. self definitional [Section 3.4, Eq. (4) and Section 4.3]
    "By activating a sparse subset of parameters per token, MoE architectures decouple generation length from total FLOP consumption. This allows them to generate significantly longer reasoning chains while remaining within the same computational budget."

    Eq. (4) defines MoE per-token FFN FLOPs as k·F_FFN + 2dE, counting only the k active experts plus router and excluding the other E−k experts. The Section 4.3 sentence 'MoE architectures decouple generation length from total FLOP consumption' is not an independently measured finding; it is the same equation asserted as a result. Since the Pareto frontier in Figure 2 and the MoE-dominance claim are computed using F(P,G) built from this estimator, the 'architectural advantage' is partly fixed by the cost definition rather than demonstrated against a measure that includes total expert parameters or serving overhead. Accuracy values are measured independently, so the circularity is partial.

full rationale

The accuracy measurements, benchmark scores, and trace-length asymmetry are independent empirical observations, and no fitted parameter is recycled into a prediction. However, the paper's central MoE conclusion rests on a FLOPs estimator whose MoE term, Eq. (4), is defined as active experts plus router. The Section 4.3 claim that sparsity 'decouples generation length from total FLOP consumption' is a restatement of that definition, so the headline architectural advantage is partially built into the metric rather than fully tested. There are no load-bearing self-citations or uniqueness-theorem arguments. The dense-vs-MoE model-pool confound (MoE entries are recent reasoning-tuned models while dense entries include older general instruct models) is a correctness/confounding concern, not a circularity, and is not scored here.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The central claims rest on an unvalidated FLOPs proxy, a single-sample evaluation without error bars, and external protocol choices (AA prompts, gpt-5.1-mini judge). No fitted constants are recycled into the accuracy claims, but the MoE efficiency conclusion is sensitive to the active-expert FLOPs assumption.

free parameters (3)
  • frequency_penalty = 0.05
    Chosen by authors after empirical trial to suppress repetitive loops; affects generated traces and therefore both accuracy and FLOPs estimates.
  • judge_truncation_window = 20 tokens
    Only the final 20 tokens of each trace are passed to the LLM judge; if answers are not contained in that window, accuracy estimates shift.
  • judge_model = gpt-5.1-mini
    The fallback judge choice affects answer verification; no judge-error study is reported.
axioms (4)
  • domain assumption FLOPs scale with parameter counts per token (2N rule and extensions)
    Eqs. (1)-(7) assume compute is dominated by matrix multiplications and ignore memory bandwidth, communication, batching, and hardware utilization; central to all Pareto comparisons.
  • domain assumption MoE per-token cost equals k active experts plus router
    Eq. (4) charges only active experts; all-to-all communication and memory-bandwidth costs of MoE serving are not counted, which can favor MoE in the headline result.
  • domain assumption Single completion per query is representative
    One sample per query at T=0.6 or greedy is used to estimate accuracy; with small benchmarks and no confidence intervals, sampling noise can change Pareto rankings.
  • domain assumption The five benchmarks are a representative difficulty spectrum
    The saturation-knee and temporal-trend conclusions are tied to this specific suite and may not generalize to other domains.
invented entities (2)
  • Efficiency score S_eff no independent evidence
    purpose: Quantifies accuracy per log10(FLOPs) for the temporal progress trend
    A new ledger entry defined in Eq. (8); it has no external benchmark or validation, and the rising trend over time is partly a property of how the metric is constructed.
  • Deductive horizon no independent evidence
    purpose: A term for the model's ceiling beyond which extra inference compute does not improve accuracy
    Conceptual limit inferred from the flattening frontier; not directly measured or independently evidenced in the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 11721 in / 12176 out tokens · 130364 ms · 2026-08-03T13:14:46.298280+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Models." pith.science (2026). https://pith.science/paper/DHNEOZB3

@misc{pith2026251224776,
  author       = {Pith},
  title        = {Pith review of: Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHNEOZB3}},
  note         = {Machine review of arXiv:2512.24776}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) are demonstrating rapid improvements on complex reasoning benchmarks, particularly when allowed to utilize intermediate reasoning steps before converging on a final solution. However, current literature often overlooks the significant computational burden associated with generating long reasoning sequences. For industrial applications, model selection depends not only on raw accuracy but also on resource constraints and inference costs. In this work, we conduct a test-time-compute aware evaluation of both contemporary and older open-source LLMs, mapping their Pareto frontiers across math- and reasoning-intensive benchmarks. Our findings identify the Mixture of Experts (MoE) architecture as a strong candidate to balance performance and efficiency in our evaluation setting. Furthermore, we trace the trajectory of Pareto efficiency over time to derive an emergent trend regarding accuracy gain per unit of compute. Finally, we demonstrate that there is a saturation point for inference-time compute. Beyond a certain threshold, accuracy gains diminish, indicating that while extended reasoning capabilities are beneficial, they cannot overcome intrinsic model limitations regarding specific complexities.

Figures

Figures reproduced from arXiv: 2512.24776 by \'Akos Prucs, M\'ark Marosi, M\'aty\'as Antal, Nara Csutora.

Figure 1
Figure 1. Figure 1: The Reasoning Pareto Frontier. Each point represents the efficiency of a model averaged across the five benchmarks. The x-axis shows the average KV-aware estimation of FLOPs per query (log scale), and the y-axis shows task accuracy. Color indicates model type (dense in blue, mixture of experts in yellow), and marker size is proportional to the total model parameter count. latency requirements, hardware lim… view at source ↗
Figure 2
Figure 2. Figure 2: The Reasoning Pareto Frontier. Each point represents the efficiency of a model on the given benchmark. The x-axis denotes the average FLOP estimation per query (log scale), while the y-axis tracks task accuracy. Color distinguishes model architecture (Dense: blue, MoE: yellow), and marker size is proportional to parameter count. Note the task-dependent saturation points, particularly visible in the lighter… view at source ↗
Figure 3
Figure 3. Figure 3: Temporal Progress. Efficiency score (average accuracy across all five benchmarks nor￾malized by log10(FLOPs)) versus model release date. The trend indicates a consistent improvement in reasoning efficiency, with increasing variance in 2025 driven by specialized reasoning models. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Trace Length Asymmetry. Average compute cost of incorrect versus correct traces. Points above the red dashed line (y = x) indicate models that spend more compute when failing. 97% of evaluated models exhibit this inefficiency. 4.5 Computational Cost of Failure Shown on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FLOPs distribution across architectural variants at sequence length 4096. GQA reduces [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: FLOPs distribution versus sequence length for Llama-3-8B (GQA-4), Llama-3-70B [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 20 linked inside Pith

  1. [1]

    Phi-4 technical report.arXiv preprint arXiv:2412.00000,

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ondrej Bajgar, Hrishikesh Baral, Paul Barton, Yonatan Bisk, Daniel Campos, et al. Phi-4 technical report.arXiv preprint arXiv:2412.00000,

    arXiv

  2. [7]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer

    URLhttps://arxiv.org/abs/ 2501.12948. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int8(): 8-bit ma- trix multiplication for transformers at scale.arXiv preprint arXiv:2208.07339,

    Pith/arXiv arXiv

  3. [8]

    9 Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letak, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al

    URL https://arxiv.org/abs/2208.07339. 9 Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letak, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  4. [9]

    Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323,

    Elias Frantar and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323,

    Pith/arXiv arXiv

  5. [11]

    Gemma Team and Google DeepMind

    URLhttps://arxiv.org/abs/2403.05530. Gemma Team and Google DeepMind. Gemma 3: Open models based on gemini research and technology.arXiv preprint arXiv:2502.00000,

    Pith/arXiv arXiv

  6. [12]

    Harvard-mit mathematics tournament (hmmt) february 2025.https://www.hmmt.org/,

    Harvard-MIT Mathematics Tournament. Harvard-mit mathematics tournament (hmmt) february 2025.https://www.hmmt.org/,

    2025

  7. [13]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hen- nigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sif...

    Pith/arXiv arXiv

  8. [14]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B

    URLhttps: //arxiv.org/abs/2505.14733. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361,

    arXiv 2001

  9. [15]

    Yungi Kim, Sunghee Song, and Bumjun Jung

    URLhttps://arxiv.org/abs/2001.08361. Yungi Kim, Sunghee Song, and Bumjun Jung. Parameter vs. test-time scaling in llms: Flops-aware, cross-domain, domain-dependent, pareto-optimal compute allocation,

    Pith/arXiv arXiv 2001

  10. [16]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe

    URL https://arxiv.org/abs/2309.06180. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023a. URLhttps://arxiv.org/abs/2305.20050. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker,...

    Pith/arXiv arXiv

  11. [17]

    Mathematical Association of America

    URLhttps: //arxiv.org/abs/2306.00978. Mathematical Association of America. American invitational mathematics examination (aime) 2025.https://www.maa.org/math-competitions/aime,

    Pith/arXiv arXiv 2025

  12. [18]

    Nemotron-nano-v2: Scalable mamba-transformer hybrids.arXiv preprint arXiv:2505.00000,

    10 NVIDIA. Nemotron-nano-v2: Scalable mamba-transformer hybrids.arXiv preprint arXiv:2505.00000,

    arXiv

  13. [19]

    Show your work: Scratchpads for intermediate computation with language models.arXiv preprint arXiv:2112.00114,

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models.arXiv preprint arXiv:2112.00114,

    Pith/arXiv arXiv

  14. [20]

    Accessed: 2024-09-12

    URLhttps://openai.com/index/ openai-o1-system-card/. Accessed: 2024-09-12. OpenAI. Gpt-oss: Open source scaling for reasoning.arXiv preprint arXiv:2504.00000,

    arXiv 2024

  15. [22]

    Related work on test- time improvement

    URLhttps://arxiv.org/abs/2405.20309. Related work on test- time improvement. Qwen Team. Qwen3 technical report.arXiv preprint arXiv:2503.00000,

    Pith/arXiv arXiv

  16. [23]

    Gpqa: A graduate-level google-proof q&a bench- mark.arXiv preprint arXiv:2311.12022,

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a bench- mark.arXiv preprint arXiv:2311.12022,

    Pith/arXiv arXiv

  17. [24]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024a

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024a. URLhttps://arxiv.org/abs/2408.03314. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling...

    Pith/arXiv arXiv

  18. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V

    URLhttps: //arxiv.org/abs/2203.11171. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022a. URLhttps://arxiv. org/abs/2201.11903. Jason Wei, Xuezhi Wang,...

    Pith/arXiv arXiv

  19. [27]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D

    URLhttps://arxiv.org/abs/2305.10601. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. InAdvances in Neural Information Processing Systems (NeurIPS),

    Pith/arXiv arXiv

  20. [28]

    11 Denny Zhou, Nathanael Sch ¨arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi

    URLhttps: //arxiv.org/abs/2203.14465. 11 Denny Zhou, Nathanael Sch ¨arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models.arXiv preprint arXiv:2205.10625,

    Pith/arXiv arXiv

  21. [29]

    6 Appendix Architectural Configurations.Table 1 summarizes representative model configurations

    URL https://arxiv.org/abs/2205.10625. 6 Appendix Architectural Configurations.Table 1 summarizes representative model configurations. Note the significant variation in the expansion factord ff/d: ranging from 1.00 (gpt-oss-20b-reason) to 8.00 (gemma-3n-E4B-it), which directly affects the FFN share of total compute and memory bandwidth requirements. Table ...

    Pith/arXiv arXiv 2048

  22. [2020]

    URL https://arxiv.org/abs/2005.14165. Tom B. Brown et al. Language modeling is compression.arXiv preprint arXiv:2402.10xxx,

    Pith/arXiv arXiv 2005

  23. [2022]

    DeepSeek-AI

    URLhttps://arxiv.org/abs/2205.14135. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.00000,

    Pith/arXiv arXiv

  24. [2023]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021a

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021a. URLhttps://arxiv.org/abs/2110.14168. Karl Cobbe, Vineet Kosaraju, Mohammad Bavari...

    Pith/arXiv arXiv

  25. [2024]

    Longlora: Efficient fine-tuning of long-context large language models.arXiv preprint arXiv:2309.12307,

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models.arXiv preprint arXiv:2309.12307,

    Pith/arXiv arXiv