Pith. sign in

REVIEW 5 major objections 6 minor 65 references

CoT-X compresses a large model's chain of thought into a short, coherent rationale so smaller models can answer accurately under tight token budgets, with up to 40.5% accuracy gains over truncation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 23:25 UTC pith:SCABQJXB

load-bearing objection Genuinely new 64-pair transfer evaluation, but the headline 40.5% gain and power-law finding are overstated by an uneven baseline and internally inconsistent numbers. the 5 major comments →

arxiv 2511.05747 v4 pith:SCABQJXB submitted 2025-11-07 cs.AI

CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization

Ziqian Bi , Yinzhi Wang , Tianyang Wang , Junfeng Hao , Benji Peng , Wenqian Weng , Jiayi Gu , Jacqueline Pang
show 1 more author
Xinyuan Song
This is my paper
classification cs.AI
keywords chain-of-thoughtcross-model transferreasoning compressionadaptive summarizationBayesian optimizationGaussian processmedical question answeringtoken budget
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long chain-of-thought reasoning helps large language models solve hard problems, but generating thousands of tokens per question is too slow and costly for smaller or latency-constrained systems. The paper tries to establish a practical alternative: have a capable thinking model produce one detailed rationale, compress that rationale into a short coherent chain under a fixed token budget, and hand the compressed chain to a cheaper answering model. On 7,501 Japanese medical licensing questions across 10 specialties, it reports up to 40.5% higher accuracy than direct truncation at the same token budget, with the biggest gains at 64–256 tokens. It also reports that a Gaussian-process Bayesian optimization layer can pick a near-optimal thinking-model/answering-model/budget configuration in 15 evaluations instead of 64, an 84% reduction in search cost. If these results hold, 'reason once, reuse many times' becomes a viable deployment pattern for reasoning-capable models under tight compute constraints.

Core claim

CoT-X claims that what transfers across models is the structure of reasoning, not the raw length of a chain. Its summarizer scores each segment of a thinking model's trace on reasoning depth, knowledge density, logical connectivity, and conclusion relevance, propagates scores through a dependency graph, greedily selects segments within the token budget, and rebuilds a coherent compressed rationale. On 7,501 Japanese medical licensing questions spanning 10 specialties, the compressed chain beats direct truncation at every budget, up to 40.5% at 64 tokens, and never loses to truncation. Intra-family transfers are strongest, but cross-family pairs become viable once compression normalizes the t

What carries the argument

The engine is a hierarchical compression pipeline: semantic segmentation of the trace into units; a composite importance score per unit combining reasoning depth, knowledge density, logical connectivity, and conclusion relevance; PageRank-style propagation over a dependency graph; budget-aware greedy selection; and coherence reconstruction that bridges gaps, preserves entities, and verifies the chain still supports the conclusion. A Gaussian-process surrogate with expected-improvement acquisition then searches over thinking model, answering model, token budget, and compression strategy, replacing a 64-point exhaustive grid with 15 evaluations.

Load-bearing premise

The load-bearing premise is that the summarizer's importance scores preserve the reasoning steps that cause the answer, not just the answer itself; the paper leaves the summarizer prompts and outputs unspecified (Section 4.3, Appendix A.3), so the 40.5% edge over truncation depends on that distinction holding.

What would settle it

Run CoT-X on the same 7,501 questions with a 64-token budget and compare against a baseline that receives just the question plus the summarizer's reconstructed conclusion under the same budget. If that baseline matches CoT-X's accuracy, the headline gain is explained by conclusion preservation rather than preserved reasoning structure, and the central claim would fail; a second check is to inspect the compressed traces and verify that the retained segments are the causal middle steps, not just the final sentence.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Reusing compressed reasoning is a zero-training transfer path: any off-the-shelf answer model can take a cached compressed chain, so small edge models can inherit reasoning ability without fine-tuning.
  • At budgets between 64 and 256 tokens, adaptive summarization beats truncation, and at higher budgets it is no worse; practitioners under latency caps should prefer semantic compression.
  • A large thinking model paired with a medium answering model can reach about 90% of top accuracy at roughly 60% less compute, making asymmetric deployment a defensible default.
  • Model selection can be re-optimized cheaply: with 15 evaluations, near-optimal configurations are found, so deployment can be re-tuned as models or budgets change.
  • The reported power-law CV ≈ 0.42 × Acc^−2.3 gives a quantitative prediction: doubling accuracy reduces cross-specialty variability by roughly a factor of five, which can guide acceptance thresholds.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the 40.5% gain is mostly about keeping the conclusion rather than preserving intermediate reasoning, then the method is effectively an answer-extraction wrapper; an ablation comparing CoT-X with a 'question plus final answer only' prompt at the same budget would separate those explanations.
  • The same compression pipeline should transfer to non-multiple-choice reasoning benchmarks such as math or open-ended medical cases; a testable prediction is that the benefit of summarization over truncation shrinks as the answer requires more unstated steps.
  • Because the summarizer itself is a 32B model, the framework amortizes well only when one long trace is reused many times; an editor-level extension would budget the summarizer's cost into the optimizer rather than treating it as free.
  • The power-law trade-off, if it generalizes, gives a simple design rule: choose the smallest configuration whose estimated accuracy clears the target on the fitted curve, then verify on worst-case specialties rather than averages.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes CoT-X, an adaptive framework for compressing long Chain-of-Thought traces produced by a large 'thinking' model and reusing the compressed trace as context for a smaller 'answering' model. The pipeline has three stages: semantic segmentation with a weighted importance score (reasoning depth, knowledge density, logical connectivity, conclusion relevance), budget-aware greedy selection on a PageRank-propagated dependency graph, and coherence reconstruction that preserves entities and the final conclusion. A Gaussian Process Bayesian optimization layer selects thinking–answering model pairs and token budgets. Experiments on 7,501 Japanese medical licensing questions, with additional Chinese/English translations, cover 64 model pairs from DeepSeek-R1 and Qwen3 families (1.5B–32B). The paper reports up to 40.5% accuracy gain over direct truncation at a 64-token budget, an 84% reduction in evaluation cost via BO, and a power-law relation CV = 0.42·Acc^(-2.3).

Significance. If the headline claims hold, the paper makes a practical contribution: zero-training, content-level CoT transfer across model families and scales, with explicit token-budget control. The scale of the evaluation (7,501 questions, 10 specialties, 3 languages, 64 model pairs) and the transparent performance matrices are notable strengths. The limitations section is candid about fixed temperatures and heuristic weights. However, the main accuracy comparison does not currently separate the effect of preserving the final conclusion from the effect of preserving reasoning structure. The power-law coefficients stated in the text contradict the numbers printed in Figure 12. The Bayesian optimization saving is computed against an incomplete exhaustive-search space. These issues are load-bearing for the central claims, so the manuscript needs major revision before the results can be accepted.

major comments (5)
  1. [§5.2, Eq. (2), §3.2.1–3.2.2] The headline 40.5% improvement over direct truncation is an apples-to-oranges comparison. Eq. (2) gives weight α4=0.25 to conclusion relevance C(s_i); §3.2.1 states that at 64 tokens 'only the conclusion and key evidence (top 5% of segments) are retained'; and §3.2.2 validates that the compressed trace still supports the final conclusion, generating 'minimal summary statements' if evidence is missing. The comparison baseline, labeled 'front' in Figures 25–27, is front-truncation of a 500–800 token trace, so the first 64 tokens typically contain problem restatement and early analysis but not the answer. Thus CoT-X is allowed to retain or regenerate the conclusion while the baseline is structurally prevented from doing so. The paper should add a conclusion-matched baseline (e.g., front truncation plus the final sentence, or 'last-64-tokens' baseline) and report the improvement attributable
  2. [§3.4, §5.3, Figure 12] The power-law coefficients in the text are inconsistent with the reported figure. The text states α≈0.42 and β≈−2.3 (also repeated in §6.1 and §7), but Figure 12's legend reports 'Pareto: CV = 0.057 × Acc^-1.01' and 'Typical (75%): CV = 0.079 × Acc^-1.27'. These are very different curves. Since the paper claims a universal power-law performance–robustness relation, the exact fitted coefficients matter; the discrepancy undermines the reproducibility of that claim. The authors should reconcile the text and figure and report the fitting procedure with uncertainty intervals.
  3. [§5.5] The claimed 84% evaluation-cost saving is computed against an incomplete exhaustive-search space. The abstract and §5.5 compare 15 BO evaluations against 'all 64 pairs' (i.e., 8×8 model pairs). But the configuration space described in §3.3 includes token budget and compression strategy as dimensions; Figure 25 lists for each thinking model at least ten answering-side configurations (front/key at five budgets plus inf). Exhaustive search over the actual space is therefore much larger than 64. Moreover, no comparison to random search or to a simple grid over budgets is reported, so the efficiency claim is not established. Please define the true exhaustive space and provide a baseline optimization method.
  4. [§4.4, §5] The evaluation section promises paired t-tests with Bonferroni correction, 95% bootstrap confidence intervals, and Cohen's d, but no such statistics appear in the results. Figure 5 shows error bars only; no p-values, confidence intervals, or effect sizes are reported anywhere for the headline comparisons. Given the emphasis on 'statistical validation' and the variability visible in the matrices, the absence of these statistics makes it impossible to tell whether the 64-token gain is statistically reliable. Please report the promised statistics, especially for the 64- and 128-token comparisons.
  5. [§4.3, Appendix A.3] The core mechanism—semantic segmentation and importance scoring—is implemented by prompting Qwen3-32B, but the prompts, few-shot exemplars, and compressed trace examples are not provided. Because Eq. (2) and §3.2.2 show that the quality of the compressed trace depends critically on what the summarizer is instructed to preserve, this is a reproducibility gap. The paper should release the prompt templates and a representative set of input-output pairs (original trace, compressed trace, and final answer) for at least one model combination. This is essential for verifying that the gain is not driven by the summarizer simply restating the answer.
minor comments (6)
  1. [§3.4] The term 'power-law relationship CV=α·Acc^β' is used as if it were a theoretical finding, but it is only a fitted empirical curve. Please clarify the distinction and report the confidence intervals on α and β.
  2. [§5.1 vs Figure 4] The text says 'reaching accuracies up to 0.85 in the best combinations', but the maximum in Figure 4's matrix is 0.791. Later, §5.3 says the best configuration has average accuracy 0.82. Please reconcile these numbers.
  3. [Figures 15–16] The histograms in Figures 15 and 16 contain unreadable mojibake/escaped unicode, so the token distribution information is not interpretable. Please regenerate these figures.
  4. [§6.3] The limitations section says the α1–α4 weights 'proved robust in sensitivity analyses', but no sensitivity analysis is shown anywhere in the paper. Please provide the analysis or remove the claim.
  5. [§5.6] The cross-lingual section reports average accuracies (55.3%, 51.4%, 51.2%) while Figure 14's caption says the overall average is 52.7%. Clarify which quantity is being averaged and how the figure's numbers relate to the text.
  6. [References] Several references appear informal or incomplete (e.g., [22] is a generic arXiv placeholder, [29] is a survey with a placeholder number). Please verify all references.

Circularity Check

1 steps flagged

Headline 40.5% gain is partly built into the definition of the compressed trace (conclusion preservation) rather than demonstrated by the comparison.

specific steps
  1. self definitional [Section 3.2.1, Section 3.2.2, Section 5.2/Figure 5]
    "C(s_i) quantifies conclusion relevance (proximity to the final answer). ... At 64 tokens, only the conclusion and key evidence (top 5% of segments) are retained. ... Validate that the compressed reasoning still supports the final conclusion. Missing evidence triggers either inclusion of additional segments or generation of minimal summary statements. ... summarization achieves an average accuracy of 0.52 versus 0.37 for truncation, a 40.5% relative improvement."

    Equation (2) explicitly weights conclusion relevance C(s_i), and Section 3.2.2 guarantees the compressed trace supports the final conclusion, even generating summary statements if needed. The comparison baseline is direct front truncation, which at 64 tokens cannot contain the conclusion of a 500–800 token reasoning trace. Thus the headline improvement is substantially a definitional consequence: CoT-X is allowed to retain or regenerate the answer while the baseline is structurally prevented from doing so. With no conclusion-matched truncation baseline or released compressed traces, the gain attributable to the importance-scoring/coherence machinery is not identified.

full rationale

The paper's methodology is not equation-level circular: the compression pipeline, Bayesian optimization efficiency, and power-law fits are empirically computed rather than assumed as premises. The only self-citation, [42], appears in Related Work and is not load-bearing. However, the central accuracy claim is confounded by construction. The compressed trace is defined, via Eq. (2) and Section 3.2.2, to preserve or regenerate the final conclusion, while the direct-truncation baseline at 64 tokens cannot contain that conclusion. The reported 40.5% improvement therefore measures, at least in part, the definitional choice to keep the answer rather than the value of the scoring/coherence machinery. This is a partial circularity in the evaluation of the headline claim, not a full equivalence: the method could still fail despite conclusion retention, and no conclusion-matched baseline or ablation is reported. Score 6 reflects one central 'prediction' that is substantially forced by the experimental setup.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

No new physical or formal entities are introduced. CoT-X, the importance scores, and the GP surrogate are methodological constructs, not entities with independent falsifiable handles. The load-bearing assumptions are the reliability of the Qwen3-32B summarizer, the fairness of retaining conclusions as a compression objective, and the validity of truncation as a baseline.

free parameters (6)
  • Importance weights α1–α4 = 0.30, 0.20, 0.25, 0.25
    Equation (2): set heuristically by hand; sensitivity is claimed in §6.3 but no sensitivity analysis is shown. These weights directly determine which segments survive compression.
  • Power-law coefficients α, β for CV = α·Acc^β = α≈0.42, β≈−2.3 in text; Fig. 12 Pareto gives α≈0.057, β≈−1.01
    Fitted to the paper’s own 64 configuration points via log-linear regression (§3.4). Text and figure disagree, so the reported law is not internally consistent.
  • Retention thresholds (top segment percentages) = 64 tokens→5%, 128→15%, 256→30%, 512→50%, 1024→75%
    Hand-specified budget-to-coverage mapping in §3.2.1 and Appendix A.3.3; no ablation is provided.
  • GP kernel hyperparameters (σ², length scale ℓ) = not reported
    The Matérn kernel in Eq. (6) has learned hyperparameters, but their values are not given, so the BO component cannot be reproduced.
  • PageRank damping factor d = 0.85
    Equation (3) fixes d = 0.85; this is a standard value but still a modeling choice that affects propagated importance scores.
  • Temperature settings = 0.7 thinking, 0.1 answering, 0.3 summarizer
    Chosen by hand; the authors say preliminary tests showed minimal impact (§6.3) but show no results.
axioms (4)
  • ad hoc to paper Qwen3-32B can reliably segment and score semantic importance of reasoning chains without task-specific training
    Appendix A.3 delegates segmentation and scoring to Qwen3-32B; no validation of scoring quality against human or expert importance judgments is provided.
  • ad hoc to paper Preserving and validating the final conclusion is an acceptable compression target
    Eq. (2) includes C(s_i) = “conclusion relevance (proximity to the final answer)” and §3.2.2 requires the compressed chain to support the final conclusion. If this is considered answer leakage, the comparison against truncation breaks.
  • ad hoc to paper Direct truncation is a meaningful baseline for CoT compression
    Section 5.2 compares only against front truncation, which discards the end of the trace where the conclusion typically sits. This is the weakest possible baseline and does not control for answer retention.
  • ad hoc to paper The 64 self-generated configurations suffice to reveal a universal power-law trade-off
    Section 3.4 fits CV = α·Acc^β on the paper’s own 64 points with no held-out validation, and the coefficients differ between the text and Figures 12/23/24.

pith-pipeline@v1.3.0-alltime-deepseek · 49180 in / 19001 out tokens · 165931 ms · 2026-08-03T23:25:48.044288+00:00 · methodology

0 comments
read the original abstract

Long Chain-of-Thought (CoT) traces can improve reasoning accuracy, but repeatedly generating them is costly for smaller or latency-constrained language models. This paper studies a practical alternative: produce a rich rationale once with a capable \emph{thinking} model, compress it, and reuse the compressed trace as context for a cheaper \emph{answering} model. We introduce CoT-X, an adaptive framework for cross-model CoT transfer. CoT-X segments reasoning traces into semantic units, scores their diagnostic and logical importance, selects budget-feasible evidence paths, and reconstructs a coherent compressed rationale for the answering model. On $7,501$ Japanese medical licensing questions spanning $10$ specialties, CoT-X improves accuracy over direct truncation by up to $40.5\%$ under the same token budget, with the largest gains at $64$--$256$ tokens. Across $64$ thinking--answering pairs from eight DeepSeek-R1 and Qwen3 models (1.5B--32B parameters), reasoning transfer is most reliable within a model family, yet remains effective across families once compression normalizes the trace. A Gaussian Process Bayesian optimization layer finds near-optimal model--budget configurations with $15$ evaluations rather than an exhaustive search over all $64$ pairs, reducing evaluation cost by $84\%$. These results show that reasoning quality, token budget, and model compatibility can be optimized jointly, making CoT-style reasoning more practical under realistic deployment constraints.

Figures

Figures reproduced from arXiv: 2511.05747 by Benji Peng, Jacqueline Pang, Jiayi Gu, Junfeng Hao, Tianyang Wang, Wenqian Weng, Xinyuan Song, Yinzhi Wang, Ziqian Bi.

Figure 1
Figure 1. Figure 1: Overview of inference paradigms. (a–b) Conventional approaches either rely on single-model end-to-end inference or [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of CoT transfer with adaptive summariza [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Hierarchical compression framework for chain-of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: 8×8 transfer matrix showing performance for all [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between adaptive summarization and di [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Improvement heatmap showing performance gains of adaptive summarization over direct truncation for all model com [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comprehensive efficiency curves showing perfor [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 10
Figure 10. Figure 10: Parallel coordinates plot showing each model’s best [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Performance heatmap of the top 20 model combi [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Scatter plot illustrating the trade-off between average performance and cross-domain robustness (measured by coefficient [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Model generation speeds measured on an 8×H100 [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Performance comparison across Japanese, Chinese, [PITH_FULL_IMAGE:figures/full_fig_p014_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Question token distribution across medical specialties. The histograms show percentage frequency with mean, median, [PITH_FULL_IMAGE:figures/full_fig_p020_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Model-generated reasoning chain token distributions by model. The histograms reveal substantial variation in reasoning [PITH_FULL_IMAGE:figures/full_fig_p020_16.png] view at source ↗
Figure 19
Figure 19. Figure 19: English dataset: Performance analysis across differ [PITH_FULL_IMAGE:figures/full_fig_p022_19.png] view at source ↗
Figure 18
Figure 18. Figure 18: Chinese dataset: Performance across medical spe [PITH_FULL_IMAGE:figures/full_fig_p022_18.png] view at source ↗
Figure 20
Figure 20. Figure 20: English dataset: Performance heatmap of model [PITH_FULL_IMAGE:figures/full_fig_p022_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Chinese dataset: Efficiency curves showing perfor [PITH_FULL_IMAGE:figures/full_fig_p023_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: English dataset: Efficiency trajectories across differ [PITH_FULL_IMAGE:figures/full_fig_p023_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Chinese dataset: Trade-off between average performance and cross-domain robustness (measured by coefficient of [PITH_FULL_IMAGE:figures/full_fig_p024_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: English dataset: Trade-off between average performance and cross-domain robustness (measured by coefficient of [PITH_FULL_IMAGE:figures/full_fig_p025_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Performance matrix showing accuracy across all thinking–answering model combinations under different token budgets [PITH_FULL_IMAGE:figures/full_fig_p026_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Performance matrix for the Chinese dataset show [PITH_FULL_IMAGE:figures/full_fig_p027_26.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 34 linked inside Pith

  1. [1]

    Chain- of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain- of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems (NeurIPS), 2022. [Online]. Available: https: //arxiv.org/abs/2201.11903

    Pith/arXiv arXiv 2022

  2. [2]

    Large language models are zero-shot reasoners,

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa, “Large language models are zero-shot reasoners,” inAdvances in Neural Information Processing Systems (NeurIPS), 2023. [Online]. Available: https: //arxiv.org/abs/2205.11916

    Pith/arXiv arXiv 2023

  3. [4]

    Towards mathematical reasoning in large language models: A survey,

    C. Ling, S. Zhou, Z. Sun, Q. Liu, and L. Zhao, “Towards mathematical reasoning in large language models: A survey,”arXiv preprint arXiv:2309.07932, 2023. [Online]. Available: https://arxiv.org/abs/2309.07932

    Pith/arXiv arXiv 2023

  4. [5]

    Med- cot: Enabling medical diagnosis reasoning via chain-of- thought fine-tuning,

    Y . Chen, P. Zhang, Y . Zhang, Z. Sun, and X. Wang, “Med- cot: Enabling medical diagnosis reasoning via chain-of- thought fine-tuning,”arXiv preprint arXiv:2310.07096,

    Pith/arXiv arXiv

  5. [6]

    Bio- cot: Biomedical chain-of-thought benchmark for large language models,

    T. Sun, H. Chen, P. Guo, Y . Liu, and J. Zhao, “Bio- cot: Biomedical chain-of-thought benchmark for large language models,”arXiv preprint arXiv:2402.00663,

    Pith/arXiv arXiv

  6. [7]

    Ed-cot: Enhancing educational question answering with chain-of-thought reasoning,

    R. Wang, Y . Zhao, X. Luo, and W. Li, “Ed-cot: Enhancing educational question answering with chain-of-thought reasoning,”arXiv preprint arXiv:2401.05672, 2024. [Online]. Available: https://arxiv.org/abs/2401.05672

    Pith/arXiv arXiv 2024

  7. [8]

    Cot- teacher: Teaching small models to reason via chain- of-thought distillation in educational tasks,

    Z. Li, H. Deng, W. Zhang, and L. Zhao, “Cot- teacher: Teaching small models to reason via chain- of-thought distillation in educational tasks,”arXiv preprint arXiv:2403.11245, 2024. [Online]. Available: https://arxiv.org/abs/2403.11245

    Pith/arXiv arXiv 2024

  8. [9]

    Lean-lm: An automated theorem prover with language models in lean,

    K. Yang, M. N. Rabe, Y . Wu, and C. Szegedy, “Lean-lm: An automated theorem prover with language models in lean,”arXiv preprint arXiv:2306.03097, 2023. [Online]. Available: https://arxiv.org/abs/2306.03097

    Pith/arXiv arXiv 2023

  9. [10]

    Proofnet: Autoformalizing and proving mathematical theorems using large language models,

    Y . Wu, Z. Wang, M. N. Rabe, and C. Szegedy, “Proofnet: Autoformalizing and proving mathematical theorems using large language models,”arXiv preprint arXiv:2402.02560, 2024. [Online]. Available: https: //arxiv.org/abs/2402.02560

    Pith/arXiv arXiv 2024

  10. [11]

    Deepseek-r1: Incentivizing reasoning ca- pability in llms via reinforcement learning,

    Z. Li, Y . Ren, H. Yuan, Y . Liu, X. Zhao et al., “Deepseek-r1: Incentivizing reasoning ca- pability in llms via reinforcement learning,”arXiv preprint arXiv:2412.19437, 2024. [Online]. Available: https://arxiv.org/abs/2412.19437

    Pith/arXiv arXiv 2024

  11. [12]

    Qwen3 technical report,

    Alibaba DAMO Academy, “Qwen3 technical report,” https://qwenlm.github.io/blog/qwen3/, 2025

    2025

  12. [13]

    Medreasoner: Large language models for medical reasoning via explicit symptom analysis and knowledge-guided chain- of-thought,

    Y . Liu, Y . Chen, M. Wang, and L. Zhao, “Medreasoner: Large language models for medical reasoning via explicit symptom analysis and knowledge-guided chain- of-thought,”arXiv preprint arXiv:2403.01234, 2024. [Online]. Available: https://arxiv.org/abs/2403.01234

    Pith/arXiv arXiv 2024

  13. [14]

    Med-cot 2.0: Evidence-aware chain-of-thought for reliable medical diagnosis,

    R. Wang, X. Zhao, K. Xu, W. Li, and X. Wang, “Med-cot 2.0: Evidence-aware chain-of-thought for reliable medical diagnosis,”arXiv preprint arXiv:2405.06278, 2024. [Online]. Available: https://arxiv.org/abs/2405.06278

    Pith/arXiv arXiv 2024

  14. [15]

    Llm- medagent: Large language model based medical treatment recommendation and clinical reasoning,

    H. Zhang, Q. Li, Q. Liu, and L. Zhao, “Llm- medagent: Large language model based medical treatment recommendation and clinical reasoning,”arXiv preprint arXiv:2404.08796, 2024. [Online]. Available: https: //arxiv.org/abs/2404.08796

    Pith/arXiv arXiv 2024

  15. [17]

    Let’s think step by step: Capturing reasoning processes in large language models,

    S. Lightman, A. Efrat, T. Scialom, S. Narang, and C. Raffel, “Let’s think step by step: Capturing reasoning processes in large language models,”arXiv preprint arXiv:2308.08708, 2023. [Online]. Available: https://arxiv.org/abs/2308.08708

    Pith/arXiv arXiv 2023

  16. [18]

    Self- consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Self- consistency improves chain of thought reasoning in language models,”arXiv preprint arXiv:2203.11171,

    Pith/arXiv arXiv

  17. [19]

    Edge-cot: Efficient chain-of-thought reasoning for edge and mobile inference,

    R. Wang, K. Xu, Y . Zhao, Z. Sun, and L. Zhao, “Edge-cot: Efficient chain-of-thought reasoning for edge and mobile inference,”arXiv preprint arXiv:2407.01823, 2024. [Online]. Available: https://arxiv.org/abs/2407.01823

    Pith/arXiv arXiv 2024

  18. [20]

    Edgegpt: Optimizing large language model inference on edge devices,

    A. Mittal, R. Jain, A. Gupta, and A. Bhattacharya, “Edgegpt: Optimizing large language model inference on edge devices,” inProceedings of the IEEE/ACM 16 Symposium on Edge Computing (SEC), 2023. [Online]. Available: https://arxiv.org/abs/2310.12761

    Pith/arXiv arXiv 2023

  19. [21]

    Available: https://arxiv.org/abs/2203

    [Online]. Available: https://arxiv.org/abs/2203. 11171

  20. [22]

    Distilling reasoning capabilities in lan- guage models via chain-of-thought transfer,

    J. Ho, T. Nguyen, T. Chen, and J. Sohl- Dickstein, “Distilling reasoning capabilities in lan- guage models via chain-of-thought transfer,”arXiv preprint arXiv:2403.02997, 2024. [Online]. Available: https://arxiv.org/abs/2403.02997

    Pith/arXiv arXiv 2024

  21. [23]

    Federated-cot: Distributed chain-of-thought collaboration across cloud and edge models,

    H. Jiang, Z. Wang, W. Li, and L. Zhao, “Federated-cot: Distributed chain-of-thought collaboration across cloud and edge models,”arXiv preprint arXiv:2312.09142,

    Pith/arXiv arXiv

  22. [24]

    Teaching small language models to reason,

    T. Magister, L. Melas-Kyriazi, T. Scialom, A. Sordoni, A. Severyn, and S. Narang, “Teaching small language models to reason,”arXiv preprint arXiv:2305.10427,

    Pith/arXiv arXiv

  23. [25]

    Available: https://arxiv.org/abs/2305

    [Online]. Available: https://arxiv.org/abs/2305. 10427

  24. [26]

    C. E. Rasmussen and C. K. I. Williams,Gaussian Processes for Machine Learning. Cambridge, MA, USA: MIT Press, 2006. [Online]. Available: http: //www.gaussianprocess.org/gpml/

    2006

  25. [27]

    Practical bayesian optimization of machine learning algorithms,

    J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” inAdvances in Neural Information Processing Systems (NeurIPS), 2012

    2012

  26. [28]

    Available: https://arxiv.org/abs/2312

    [Online]. Available: https://arxiv.org/abs/2312. 09142

  27. [29]

    Language models are few-shot learners,

    T. Brownet al., “Language models are few-shot learners,” inNeurIPS, 2020

    2020

  28. [30]

    The pagerank citation ranking: Bringing order to the web,

    L. Page, S. Brin, R. Motwani, and T. Winograd, “The pagerank citation ranking: Bringing order to the web,” inProceedings of the 7th International World Wide Web Conference (WWW), 1999. [Online]. Available: http://ilpubs.stanford.edu:8090/422/

    1999

  29. [31]

    Scaling laws for transfer,

    D. Hernandez, J. Kaplan, T. Henighan, and S. McCandlish, “Scaling laws for transfer,”arXiv preprint arXiv:2102.01293, 2021. [Online]. Available: https://arxiv.org/abs/2102.01293

    Pith/arXiv arXiv 2021

  30. [32]

    Training verifiers to solve math word problems,

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, J. Kaplan, A. Power, L. Knight, and W. Zaremba, “Training verifiers to solve math word problems,” in Advances in Neural Information Processing Systems (NeurIPS), 2021. [Online]. Available: https://arxiv.org/ abs/2110.14168

    Pith/arXiv arXiv 2021

  31. [33]

    A tutorial on bayesian optimization,

    P. I. Frazier, “A tutorial on bayesian optimization,”arXiv preprint arXiv:1807.02811, 2018

    Pith/arXiv arXiv 2018

  32. [34]

    Large language mod- els for summarization: A survey,

    Y . Zhang, Y . Xu, and T. Sun, “Large language mod- els for summarization: A survey,”arXiv preprint arXiv:2311.12345, 2023

    Pith/arXiv arXiv 2023

  33. [35]

    Scaling laws for neural language models,

    J. Kaplan, S. McCandlish, T. Henighanet al., “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020. [Online]. Available: https: //arxiv.org/abs/2001.08361

    Pith/arXiv arXiv 2001

  34. [36]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” inNeural Information Processing Systems Deep Learning Workshop, 2015. [Online]. Available: https://arxiv.org/abs/1503.02531

    Pith/arXiv arXiv 2015

  35. [37]

    Training data-efficient image trans- formers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablay- rolles, and H. Jegou, “Training data-efficient image trans- formers & distillation through attention,” inInternational Conference on Machine Learning (ICML), 2021

    2021

  36. [38]

    Least-to-most prompting enables complex reasoning in large language models,

    D. Zhou, L. Zhao, and X. Lin, “Least-to-most prompting enables complex reasoning in large language models,” inInternational Conference on Learning Representations (ICLR), 2023. [Online]. Available: https://arxiv.org/abs/ 2205.10625

    Pith/arXiv arXiv 2023

  37. [39]

    A survey of model compression and acceleration for deep neural networks,

    Y . Cheng, D. Wang, P. Zhou, and T. Zhang, “A survey of model compression and acceleration for deep neural networks,”arXiv preprint arXiv:1710.09282, 2017. [Online]. Available: https://arxiv.org/abs/1710.09282

    Pith/arXiv arXiv 2017

  38. [40]

    Compression of deep neural networks for deployment on edge devices: A survey,

    U. Guptaet al., “Compression of deep neural networks for deployment on edge devices: A survey,”ACM Computing Surveys, 2020

    2020

  39. [41]

    Pegasus: Pre- training with extracted gaps sentences for abstractive sum- marization,

    J. Zhang, Y . Zhao, M. Saleh, and P. Liu, “Pegasus: Pre- training with extracted gaps sentences for abstractive sum- marization,”International Conference on Machine Learn- ing (ICML), 2020

    2020

  40. [42]

    Adaptive reasoning sum- marization for efficient chain-of-thought compression,

    X. Song, Z. Sun, and L. Zhao, “Adaptive reasoning sum- marization for efficient chain-of-thought compression,” arXiv preprint arXiv:2406.07152, 2024

    Pith/arXiv arXiv 2024

  41. [43]

    A survey of text summa- rization techniques,

    A. Nenkova and K. McKeown, “A survey of text summa- rization techniques,”Foundations and Trends in Informa- tion Retrieval, vol. 5, no. 2–3, pp. 103–233, 2012

    2012

  42. [44]

    Get to the point: Summarization with pointer-generator networks,

    A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” inAn- nual Meeting of the Association for Computational Lin- guistics (ACL), 2017

    2017

  43. [45]

    Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,

    M. Lewis, Y . Liu, N. Goyalet al., “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” inAnnual Meeting of the Association for Computational Linguistics (ACL), 2020

    2020

  44. [46]

    Boprompt: Bayesian op- timization for prompt engineering in large language mod- els,

    Z. Yang, X. Liu, and L. Zhao, “Boprompt: Bayesian op- timization for prompt engineering in large language mod- els,”arXiv preprint arXiv:2405.08564, 2024

    Pith/arXiv arXiv 2024

  45. [47]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Zhao, J. Yu, and T. Chen, “Tree of thoughts: Deliberate problem solving with large language models,” arXiv preprint arXiv:2305.10601, 2023, proposes struc- tured reasoning frameworks that maintain logical flow across intermediate reasoning steps

    Pith/arXiv arXiv 2023

  46. [48]

    Neural architecture search with bayesian op- timisation and optimal transport,

    K. Kandasamy, W. Neiswanger, J. Zhang, B. Poczos, and E. P. Xing, “Neural architecture search with bayesian op- timisation and optimal transport,” inAdvances in Neural Information Processing Systems (NeurIPS), 2018. 17

    2018

  47. [49]

    Hyperband: A novel bandit-based ap- proach to hyperparameter optimization,

    L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, “Hyperband: A novel bandit-based ap- proach to hyperparameter optimization,” inInternational Conference on Learning Representations (ICLR), 2017

    2017

  48. [50]

    Taking the human out of the loop: A re- view of bayesian optimization,

    B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas, “Taking the human out of the loop: A re- view of bayesian optimization,”Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2016

    2016

  49. [51]

    Efficient serving of large language models with con- tinuous batching,

    R. Tang, T. Zhang, D. Narayanan, M. Zaharia, and I. Sto- ica, “Efficient serving of large language models with con- tinuous batching,” inProceedings of the 2024 USENIX Annual Technical Conference (USENIX ATC), 2024

    2024

  50. [52]

    Efron and R

    B. Efron and R. J. Tibshirani,An Introduction to the Boot- strap. New York, NY: Chapman & Hall, 1994

    1994

  51. [53]

    Faithful sum- marization with entity-aware decoding,

    S. Cao, Y . Dong, and J. C. K. Cheung, “Faithful sum- marization with entity-aware decoding,”Transactions of the Association for Computational Linguistics, vol. 10, pp. 879–894, 2022, introduces entity-aware mechanisms to maintain factual and referential consistency in summa- rization

    2022

  52. [54]

    A fast and elitist multiobjective genetic algorithm: Nsga-ii,

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii,” inProceedings of the IEEE Transactions on Evolution- ary Computation, vol. 6, no. 2, 2002, pp. 182–197, semi- nal work introducing NSGA-II for Pareto-optimal solution discovery in multi-objective optimization

    2002

  53. [55]

    Efficient memory management for large language model serving with pagedattention,

    J. Kwon, W. Yu, X. He, D. Narayanan, M. Zaharia, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,”arXiv preprint arXiv:2309.06180, 2023. [Online]. Available: https://arxiv.org/abs/2309.06180

    Pith/arXiv arXiv 2023

  54. [56]

    Pagedattention: Efficient memory management for large language model inference,

    W. Kwon, Z. Li, S. Zhuang, L. Zheng, and I. Sto- ica, “Pagedattention: Efficient memory management for large language model inference,” https://github.com/ vllm-project/vllm, 2023, official implementation of Page- dAttention used for KV-cache management in vLLM

    2023

  55. [57]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” inInternational Conference on Learning Representations (ICLR), 2020. [Online]. Available: https: //openreview.net/forum?id=SkeHuCVFDr

    2020

  56. [58]

    Cohen,Statistical Power Analysis for the Behavioral Sciences

    J. Cohen,Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge, 1988

    1988

  57. [59]

    Multi- lingual performance of large language models: Surpris- ing strengths and unexpected weaknesses,

    H. Liu, C. Emezue, S. Ruder, and E. M. Ponti, “Multi- lingual performance of large language models: Surpris- ing strengths and unexpected weaknesses,”Transactions of the Association for Computational Linguistics (TACL), vol. 12, pp. 233–250, 2024

    2024

  58. [60]

    vllm: Easy, fast, and cheap llm serving with pagedattention,

    vLLM Team, “vllm: Easy, fast, and cheap llm serving with pagedattention,” https://vllm.ai/, 2023, accessed: 2025- 11-07

    2023

  59. [61]

    Geneva, Switzerland: World Health Organization, 2019

    World Health Organization,International Classification of Diseases 11th Revision (ICD-11). Geneva, Switzerland: World Health Organization, 2019. [Online]. Available: https://icd.who.int/

    2019

  60. [62]

    SNOMED CT: The advanced terminology and coding system for ehealth,

    K. Donnelly, “SNOMED CT: The advanced terminology and coding system for ehealth,”Studies in Health Technol- ogy and Informatics, vol. 121, pp. 279–290, 2006. 18 A Supplementary Figures A.1 Token Distribution Analysis Understanding the distribution of tokens in both questions and reasoning chains is essential for optimizing compression strate- gies and det...

    2006

  61. [63]

    Gptscore: Evaluate as you desire,

    T. Liu, S. Xu, Y . Fu, W. X. Zhao, and J.-R. Wen, “Gptscore: Evaluate as you desire,”arXiv preprint arXiv:2302.04166, 2023. [Online]. Available: https://arxiv.org/abs/2302.04166

    Pith/arXiv arXiv 2023

  62. [64]

    Compressed chain-of- thought: Efficient reasoning through dense representa- tions,

    J. Cheng and B. Van Durme, “Compressed chain-of- thought: Efficient reasoning through dense representa- tions,”arXiv preprint arXiv:2412.13171, 2024. [Online]. Available: https://arxiv.org/abs/2412.13171

    Pith/arXiv arXiv 2024

  63. [65]

    Tokenskip: Controllable chain-of-thought compression in llms,

    H. Xia, Y . Li, C. T. Leong, W. Wang, and W. Li, “Tokenskip: Controllable chain-of-thought compression in llms,” inarXiv preprint arXiv:2502.12067, 2025. [Online]. Available: https://arxiv.org/abs/2502.12067

    arXiv 2025

  64. [2023]

    Available: https://arxiv.org/abs/2310

    [Online]. Available: https://arxiv.org/abs/2310. 07096

  65. [2024]

    Available: https://arxiv.org/abs/2402

    [Online]. Available: https://arxiv.org/abs/2402. 00663