Pith. sign in

REVIEW 3 major objections 6 minor 31 references

Real-Time Progress Prediction in Reasoning Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reasoning language models can be trained to report reasoning progress in real time, reaching 0.161 mean absolute error on math traces.

desk verdict Careful feasibility study that mostly measures how well reasoning LMs track normalized token position, with a headline result that needs error bars to hold. read the letter →

arxiv 2506.23274 v4 pith:CVFYCLPV submitted 2025-06-29 cs.LG cs.AI

Hans Peter Lyngs{o}e Raaschou-Jensen , Constanza Fierro , Anders S{o}gaard This is my paper
classification cs.LGcs.AI
keywords reasoninglanguagemodelschain-of-thoughtprogresspredictionlinearprobingsupervisedfine-tuningmeanabsoluteerrorrolloutdispersiontest-timescaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a reasoning language model can tell a user how far it has gotten while it is still thinking. The authors show three things: hidden states along the reasoning trace carry linearly recoverable information about normalized trace position; LoRA fine-tuning can make a model emit .. estimates during generation, with the best masked checkpoint reaching 0.161 mean absolute error on mathematical traces and beating token-position baselines in aggregate; and the remaining randomness in continuation length sets a floor on predictability, with mean absolute deviation between 0.027 and 0.065 across models. The practical point is that long chain-of-thought runs, which are otherwise opaque, could expose a live progress estimate for expectation management and real-time oversight.

What carries the argument

The central object is the progress ratio, adapted from search progress: $Progress(C_k) = k/(k+\mathbb{E}_{\tau\sim P_\theta(\cdot|q,C_k)}|\tau|)$, approximated in practice by $k/(k+|\tau|)$ for a sampled continuation. It maps any partial trace to $[0,1]$ and supplies the supervised labels for both the probe and the fine-tuned progress reporter. The other load-bearing pieces are: ten-bucket linear probes over hidden states (with and without question embeddings) that test linear recoverability; LoRA fine-tuning with <progressbar>...</progressbar> annotations whose tokens are upweighted by $\gamma=5$ in the loss; a cosine masking schedule that randomly hides previous annotations during training so the model cannot simply extrapolate from marker spacing; and mean absolute deviation and mean absolute percentage deviation over eight rollouts per prefix to measure how much a single prefix's true progress varies.

What would settle it

Insert a long block of fluent but semantically vacuous reasoning tokens into the middle of a solved trace and rerun the progress-reporting checkpoint; if the emitted progress values shift by roughly the inserted fraction of the trace rather than staying tied to the amount of useful reasoning remaining, the estimates are tracking token position, not progress.

Watch

Extended reading notes

Core claim

The paper's central claim is that real-time progress prediction is feasible for reasoning language models. It adapts the classical search-progress definition to a partial reasoning trace $C_k$, defining progress as $k/(k+\mathbb{E}_{\tau\sim P_\theta(\cdot|q,C_k)}|\tau|)$, the expected fraction of tokens already emitted, and uses this to label every token in a completed trace. A linear probe over hidden states classifies tokens into ten progress buckets at roughly 30% accuracy (chance 10%), with errors concentrated near the diagonal, while a fine-tuned model that writes progress annotations during generation reaches 0.161 MAE on math benchmarks and outperforms position baselines in the aggregate math comparison. The paper is careful to read the probe results as a representation diagnostic rather than proof of semantic progress, because labels derived from $k/m$ can partly reflect token-position and trace-length regularities. It also quantifies label ambiguity by sampling eight continuations per prefix: the same partial rollout implies different realized progress values, with MAD from 0.027 (Qwen3-4B) to 0.065 (Qwen3-0.6B), all below the achieved MAE.

Load-bearing premise

The load-bearing premise is that progress equals the fraction of reasoning tokens emitted so far, so a report can be numerically accurate while the genuinely useful reasoning work remaining is different.

Editorial extensions

If this is right

  • Users watching a long chain-of-thought run could see a live progress bar that is a model-generated, numerically calibrated estimate rather than a guess from elapsed time.
  • Direct progress supervision adds signal beyond length statistics in the mathematical setting, since the best progress checkpoints beat token-position baselines in the aggregate math comparison.
  • The masking schedule is the variant that gives the best progress-reporting checkpoint, showing that auxiliary reporting tokens can be trained without letting the model simply exploit marker spacing.
  • Because MAD (0.027–0.065) is lower than achieved MAE (0.161–0.294), continuation-length randomness explains only part of the error, so better progress estimators remain possible.
  • Out-of-domain BBEH traces keep progress reports parseable and numerically aligned, but strong length-statistics baselines there mean out-of-domain MAE must be read against structural controls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to retarget progress from token fraction to task-level subgoal completion, which would let a progress bar stay meaningful during backtracking and verbose padding; the paper's own Figure 6 shows such cases can break monotonicity.
  • The dispersion results suggest that progress-label stability may be partly a model-scale property; a concrete prediction not made in the paper is that a small model fine-tuned on traces distilled from a low-dispersion large model would show better calibration than when trained on its own traces.
  • A cheap diagnostic for locating the remaining error would be to fit a regression head on the same hidden states and compare its MAE with the emitted annotation values; if the head is better, the bottleneck is the generation interface rather than the encoded information.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies whether reasoning language models can predict 'progress' during chain-of-thought generation, defining progress as normalized trace position (current prefix length over completed trace length). It first trains linear probes on hidden states to classify progress into ten bins across DEEPSCALER-1.5B, QWEN3-0.6B, and QWEN3-4B, finding in-domain accuracies of roughly 25–36%. It then LoRA-fine-tunes the same models to emit <progressbar>..</progressbar> annotations, with a masking variant that removes prior annotations during training, and evaluates progress MAE on MATH500, AMC23, OlympiadBench, and BBEH. The best checkpoint (masked QWEN3-4B) achieves 0.161 MAE on mathematical reasoning traces (0.162 in the aggregate table), and the paper claims it outperforms token-position baselines. A rollout-dispersion analysis estimates the intrinsic ambiguity of progress labels by sampling eight continuations per prefix, finding the lowest dispersion for QWEN3-4B.

Significance. The paper is well structured and unusually honest: it includes a dataset-only SFT control, diagnostic position/length baselines, a matched probe-versus-SFT comparison, and an audit of the rollout jobs. If the headline comparison is statistically supported, the approach is a practical way to provide user-facing progress estimates, and the ambiguity analysis is a useful characterization of label noise. The main weakness is that the central claim of outperforming position baselines rests on a very small margin without any uncertainty quantification; the probe results also lack a position-only control that would separate content-sensitive progress from positional encoding recovery.

major comments (3)
  1. [§5.2, Conclusion; Table 8 / Appendix D] The claim that the best progress-reporting checkpoint 'outperforms token-position baselines' rests on a 0.006 MAE margin over the strongest baseline: in Appendix D, Table 8, masked QWEN3-4B achieves 0.162 aggregate mathematical MAE versus 0.168 for the previous-marker baseline, while Table 2 reports 0.161 for the same checkpoint on the mathematical benchmarks. No standard errors, confidence intervals, or multiple seeds are reported anywhere in the paper. Given the stochasticity of LoRA fine-tuning, the sampling temperature used to generate evaluation traces, and the small evaluation sets (MATH500 plus 40–120 AMC/Olympiad examples), this margin is not sufficient to establish the conclusion in Section 5.2 that direct progress supervision adds signal beyond length statistics. Please provide multi-seed reruns with confidence intervals, or a paired/bootstrap test over traces, that demonstrates the margin is robust.
  2. [§3, §5 Metrics; Figure 6] The operational definition Progress(C_k) = k / (k + E[|tau|]) and the evaluation label g_i = |C_i| / |tau| equate progress with the fraction of tokens emitted so far. The paper acknowledges this in Section 4 and in the Limitations, but the practical claim in the introduction—progress toward the final answer—requires evidence that this positional quantity tracks remaining reasoning work. Figure 6 shows backtracking examples where the predicted progress is non-monotonic, illustrating that position and useful work can diverge. Please add a concrete validation: compute the correlation between predicted progress and a task-level measure of remaining work (for example, the number of future reasoning steps needed to reach the answer, or whether the prefix contains backtracking or verbose irrelevant tokens), and report how the MAE numbers change when such cases are excluded.
  3. [§4, Table 1] The probe evidence that hidden states encode progress information would be substantially more informative with a position-only control. Because the labels are derived from k/m, a linear probe applied at token position k can succeed by exploiting positional encodings and trace-length statistics, and the 10% random-guess reference is not a strong baseline. Please add a probe or logistic-regression baseline trained on token-index features (or on the positional encoding alone) so the reader can see how much of the probe accuracy is attributable to position rather than reasoning content. This is directly relevant to the interpretation of the probe results as evidence about progress information rather than position information.
minor comments (6)
  1. [Abstract vs. Appendix D] The abstract states 0.161 MAE on mathematical reasoning traces, while Appendix D, Table 8 reports 0.162 for the aggregate mathematical benchmark comparison; please clarify which number corresponds to which aggregation.
  2. [§6.2, Table 5] The claim that larger models make progress labels more stable is based on only three models that differ in model family, training domain, and architecture (DEEPSCALER-1.5B is domain-specific while the QWEN3 models are general-purpose). Please soften the claim or provide a controlled scaling comparison.
  3. [§5.1] The masking schedule description says prior progress annotation spans are 'masked with probability rho' but does not specify whether the masked spans are replaced with a placeholder token or dropped entirely; this affects the training dynamics and should be stated.
  4. [§5.1, Eq. (3)] The progress-loss weight gamma is applied to delimiter tokens as well as progress values, but the text does not say whether the weight also applies to the closing delimiter or only to the numeric value; please clarify.
  5. [Table 2 caption] The caption should state explicitly that original and dataset-only SFT rows do not emit progress estimates and hence have no MAE, rather than leaving this to Appendix A.
  6. [§3] In the definition of Progress(C_k), the approximation with a single sampled continuation |tau| obscures the fact that the paper uses r sampled rollouts in Section 6.2; please define the Monte Carlo estimator with r in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper defines progress as normalized position, discloses the position/length confounds, and its central comparisons are empirical supervised-learning results rather than derivations from inputs.

full rationale

The paper's progress label is constructionally tied to token position: Section 3 defines Progress(C_k) ≈ k/(k+|τ|) and Section 5 evaluates g_i=|C_i|/|τ|. This makes 'progress' a normalized-position quantity. However, the paper does not present this as a first-principles derivation; it explicitly labels the probe study a 'representation diagnostic rather than conclusive evidence of semantic progress tracking, since the labels are derived from k/m and can partly reflect token-position and expected-length information' (Section 4). The SFT experiments train models to emit the same position-based annotations, which is standard supervised learning, and the claimed outperformance over baselines is an empirical comparison against position/length baselines defined in Section 5.1 and Appendix D; the baselines are stated to be 'diagnostic rather than deployable online predictors because they use realized corpus-level information' (Appendix E). The masking schedule (Section 5.1) is an explicit attempt to reduce a known shortcut ('a model may be able to extrapolate from earlier progress reports and their positions in the sequence rather than infer progress from the reasoning content itself,' Section 6.1). No load-bearing self-citation occurs; the progress formulation is attributed to Thayer et al. (2012). The principal weaknesses are statistical—the 0.006 MAE margin over the previous-marker baseline in Table 8 has no error bars—and construct-validity concerns (normalized position may not equal useful progress), but neither is an equivalence-by-construction or a fitted-parameter-renamed-as-prediction. Hence no circular step is established.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the definition of progress as normalized trace position, which conflates position with semantic completion. No physical constants or fitted scientific parameters are involved; the free parameters are training hyperparameters chosen by hand. The <progressbar> format is a textual artifact rather than a scientific entity.

free parameters (4)
  • progress-token loss weight gamma = 5
    Hand-chosen upweight for progress annotation tokens in the SFT loss (Eq. 3). Not fitted; modest effect.
  • masking schedule maximum probability rho = 0.5 with cosine schedule
    Hand-chosen regularization strength to reduce reliance on prior markers, introduced in Section 5.1.
  • number of progress bins Q = 10
    Discretization of progress for probe classification; equal bins of normalized position, defined in Section 4.
  • LoRA rank and alpha = 256
    Hand-chosen LoRA configuration used in all fine-tuning, Appendix B.
assumptions (5)
  • domain assumption Progress is defined as normalized trace position: Progress(C_k) = k / (k + E[|tau|])
    Section 3. Equates progress with the fraction of tokens emitted, not with semantic completion. Central to all experiments.
  • domain assumption The realized progress label g_i = |C_i|/|tau| is the correct ground truth for evaluating progress predictions
    Section 5, Metrics. Assumes final trace length is available at evaluation and is the right reference.
  • standard math Hidden states at position k contain linearly accessible information about k (positional encodings) and about expected trace length
    Transformer architecture; the paper relies on this for the linear probe to have any signal.
  • domain assumption Sampled continuations tau ~ P_theta(.|q, C_k) provide an unbiased estimate of the expected continuation length
    Section 3 and Section 6.2; used to approximate progress and to measure rollout dispersion.
  • domain assumption OpenR1-Math traces and MATH500/AMC23/OlympiadBench are representative of reasoning LM workloads
    Dataset choice in Sections 4 and 5; claims of generalization depend on this.
invented entities (1)
  • <progressbar> annotation token format
    purpose: Allows fine-tuned models to emit progress estimates inside the reasoning trace
    A textual marker, not a scientific entity; its utility depends on the position-based progress definition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Real-Time Progress Prediction in Reasoning Language Models." pith.science (2026). https://pith.science/paper/CVFYCLPV

@misc{pith2026250623274,
  author       = {Pith},
  title        = {Pith review of: Real-Time Progress Prediction in Reasoning Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CVFYCLPV}},
  note         = {Machine review of arXiv:2506.23274}
}
read the original abstract

Recent reasoning language models, particularly those that employ long latent chains of thought, achieve strong performance on complex agentic tasks. However, as these models operate over increasingly long time horizons, their internal progress becomes opaque to users, making expectation management and real-time oversight difficult. In this work, we investigate whether real-time progress prediction is feasible for such models. We first test whether hidden states encode progress information by discretizing reasoning trajectories and training a linear probe to classify reasoning states. We then fine-tune models to generate progress estimates from 0--100\% during chain-of-thought reasoning. Our strongest progress-reporting checkpoint reaches 0.161 MAE on mathematical reasoning traces and outperforms position baselines in this setting. Finally, we quantify the intrinsic ambiguity of progress labels by measuring how much the implied progress value varies from the same partial rollout. This ambiguity is lowest for Qwen3-4B, whose continuations produce the smallest rollout dispersion, suggesting that larger models can make progress labels more stable by reducing variation in remaining solution length.

Figures

Figures reproduced from arXiv: 2506.23274 by the authors.

Figure 1
Figure 1. Real-time progress tracking during LLM reasoning. The model intermittently updates the user-facing progressbar by using <progressbar>..</progressbar>. models to return incorrect answers despite being ca￾pable of solving the problem given a larger reasoning budget. Recent research suggests that reasoning LMs encode confidence in their hidden states before materializing a final answer. For example, Zhang et al. (2025)… view at source ↗
Figure 2
Figure 2. Probe predictions on the in-domain DEEPSCALER-1.5B test split, grouped by reasoning-trace length. Top: average predicted probability over progress buckets. Bottom: expected progress curves, with the dashed line showing perfect calibration. See Appendix A for how probabilities are computed and aggregated. The corresponding long-trace held-out visualization is shown in Appendix 10. lengths, enabling us to study progre… view at source ↗
Figure 3
Figure 3. Prediction error (MAE) versus reasoning [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Fraction of non-monotonic predictions for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: MAD (Equation 4) and MAPD (Equation 5) for QWEN3-0.6B, DEEPSCALER-1.5B, and QWEN3- 4B, computed over 50 sequence length bins. Each panel overlays the three base models using separate colors. remain parseable and numerically aligned, but strong token-position and task-l…
Figure 6
Figure 6. Figure 6: Sixteen examples of backtracking. The line with slope 1 indicates perfect prediction for the model trained [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Example progress-prediction trajectories for the unmasked progress-reporting model. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Single-trace linear probe visualization for a 12K-token sequence. Left: predicted progress-bucket [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: BBEH prediction error (MAE) versus sequence length, binned into 25 groups, across Qwen3-0.6B, [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Probe predictions on long-trace held-out [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 8 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anum Afzal, Florian Matthes, Gal Chechik, and Yftah Ziser. 2025. https://arxiv.org/abs/2505.24362 Knowing before saying: Llm representations encode information about chain-of-thought success before completion . Preprint, arXiv:2505.24362

    arXiv 2025

  4. [4]

    Pranjal Aggarwal and Sean Welleck. 2025. https://arxiv.org/abs/2503.04697 L1: Controlling how long a reasoning model thinks with reinforcement learning . Preprint, arXiv:2503.04697

    arXiv 2025

  5. [5]

    AMC. 2023. https://huggingface.co/datasets/math-ai/amc23 Amc 2023 mathematics competition problems (via hugging face dataset) . Hugging Face Datasets. Dataset created by math-ai; Accessed: October 6, 2025

    work page 2023

  6. [6]

    Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow, Atticus Geiger, Owen Lewis, and Jack Merullo. 2026. https://arxiv.org/abs/2603.05488 Reasoning theater: Disentangling model beliefs from chain-of-thought . Preprint, arXiv:2603.05488

    arXiv 2026

  7. [7]

    Kyle Cox, Darius Kianersi, and Adri \`a Garriga-Alonso. 2026. https://arxiv.org/abs/2603.01437 Decoding answers before chain-of-thought: Evidence from pre-cot probes and activation steering . Preprint, arXiv:2603.01437

    arXiv 2026

  8. [8]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement lea...

    arXiv 2025

Show all 31 references
  1. [9]

    Hugging Face. 2025. Openr1-math-220k. https://huggingface.co/datasets/open-r1/OpenR1-Math-220k

    2025

  2. [10]

    Markus Frohmann, Igor Sterner, Ivan Vuli \'c , Benjamin Minixhofer, and Markus Schedl. 2024. https://aclanthology.org/2024.emnlp-main.665 Segment any text: A universal approach for robust, efficient and adaptable sentence segmentation . In Proceedings of the 2024 Conference on...

    2024

  3. [11]

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, and 1 others. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprin...

    2024 arXiv

  4. [12]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . Preprint, arXiv:2106.09685

    2021 arXiv

  5. [13]

    Suozhi Huang, Peiyang Song, Robert Joseph George, and Anima Anandkumar. 2025. https://arxiv.org/abs/2502.17925 Leanprogress: Guiding search for neural theorem proving via proof progress prediction . Preprint, arXiv:2502.17925

    2025

  6. [14]

    Prime Intellect Team: Sami Jaghour, Justus Mattern, Jack Min Ong, Jannik Straube, Manveer Basra, Aaron Pazdera, Matthew Di Ferrante, Kushal Thaman, Felix Gabriel, Fares Obeid, Kemal Erdem, Michael Keblinger, and Johannes Hagemann. 2025. https://storage.googleapis.com/public-te...

    2025

  7. [15]

    Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q

    Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K. Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Q...

    2025 doi arXiv

  8. [16]

    Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles, Seraphina Nix, Tao Lin, Neev Parikh, and 6 others. 202...

    2025 arXiv

  9. [17]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050

    2023 arXiv

  10. [18]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/DeepScaleR-Su...

    2025

  11. [19]

    Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, and Caiming Xiong. 2019. https://arxiv.org/abs/1901.03035 Self-monitoring navigation agent via auxiliary progress estimation . Preprint, arXiv:1901.03035

    2019 arXiv

  12. [20]

    Moura and Ullrich. 2021. The lean 4 theorem prover and programming language. In Automated Deduction -- CADE 28, pages 625--635, Cham. Springer International Publishing

    2021

  13. [21]

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. https://arxiv.org/abs/2501.19393 s1: Simple test-time scaling . Preprint, arXiv:2501.19393

    2025 arXiv

  14. [22]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 oth...

    2024 arXiv

  15. [23]

    Matan Sudry and Erez Karpas. 2021. https://openreview.net/forum?id=W9O6ueH5U37 Learning to estimate search progress using sequence of states . In ICAPS 2021 Workshop on Heuristics and Search for Domain-independent Planning

    2021

  16. [24]

    Qwen Team. 2025. https://qwenlm.github.io/blog/qwen3/ Qwen3

    2025

  17. [25]

    Thayer, Roni Stern, and Levi H

    Jordan T. Thayer, Roni Stern, and Levi H. Lelis. 2012. Are we there yet? -- estimating search progress. In Proceedings of the International Symposium on Combinatorial Search (SOCS). AAAI Press

    2012

  18. [26]

    Zhihui Xie, Jizhou Guo, Tong Yu, and Shuai Li. 2024. https://arxiv.org/abs/2405.18711 Calibrating reasoning in language models with internal consistency . Preprint, arXiv:2405.18711

    2024 arXiv

  19. [27]

    Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. 2025. https://arxiv.org/abs/2504.05419 Reasoning models know when they're right: Probing hidden states for self-verification . Preprint, arXiv:2504.05419

    2025 arXiv

  20. [28]

    Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. 2020. https://arxiv.org/abs/1911.07883 Vision-language navigation with self-supervised auxiliary reasoning tasks . Preprint, arXiv:1911.07883

    2020 arXiv

  21. [29]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  22. [30]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  23. [31]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.