Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Scoring multi-agent reasoning can reuse the generator's own memory instead of re-reading the text, cutting cost from quadratic to linear while matching or beating text-based scorers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 05:02 UTC pith:THVPHY5F

load-bearing objection Clean systems fix for the quadratic PRM bottleneck in multi-agent TTS: single-token KV readout gives real O(L) scoring with matching or better accuracy under the paper's own baseline, plus usable theory. the 2 major comments →

arxiv 2607.09153 v1 pith:THVPHY5F submitted 2026-07-10 cs.AI

KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling

Peng Kuang , Haibo Jin , Xiaoyu Han , Yanli Wang , Xiaopeng Yuan , Ye Yu , Kaidi Xu , Haohan Wang This is my paper
classification cs.AI
keywords process reward modelKV cachetest-time scalingmulti-agent systemsLoRAbeam searchMCTSmathematical reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Process reward models guide multi-agent search by scoring intermediate steps, but today's scorers re-encode the full trajectory as text, paying a quadratic attention cost that becomes prohibitive on long rollouts. This paper shows that the key-value cache already written during generation is a richer representation of the same trajectory and can be scored with a single "verify" token. That changes the per-call cost from O(L^{2}) to O(L). Theory proves the cache is denser than text and that one token already captures most of the reward signal; experiments on MATH, GSM8K and AIME confirm that the resulting scorer matches or exceeds text-based PRMs under beam search, MCTS and weighted voting, while delivering thousands-fold fewer FLOPs, tens-fold lower latency and memory. The same continuous scoring also opens a path to gradient-based steering of latent messages between agents.

Core claim

The generation-time key-value cache is a strictly richer and more efficient input for process reward modeling than the decoded text. Conditioning a lightweight LoRA adapter on that cache with a single verify token yields scores that match or surpass full-text re-encoding PRMs while reducing scoring complexity from O(L^{2}) to O(L).

What carries the argument

KV-PRM: a depth-1 readout that appends one "verify" token, attends it against the frozen generation KV cache through a LoRA adapter, and reads the resulting logits over "+"/"−" judgment tokens. The mechanism turns the cache from a generation byproduct into the reward model's input representation.

Load-bearing premise

The generator and verifier must share the same architecture so the verifier can attend the native KV tensors, and the hidden states must admit a linear semantic basis for the capacity and near-optimality proofs to hold.

What would settle it

Train matched KV-PRM and text-PRM on identical MCTS labels, then measure accuracy and wall-clock scoring cost under beam search or MCTS on MATH or AIME when trajectories exceed a few thousand tokens: if KV-PRM accuracy falls materially below text-PRM while cost savings disappear, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Test-time search over long multi-agent trajectories can treat process scoring as essentially free relative to generation.
  • Verifier cost no longer forces designers to shrink the scorer or shorten trajectories, removing a hard systems bottleneck.
  • Because scores are continuous functions of the cache, gradient-based optimization of latent inter-agent messages becomes possible without decoding to text.
  • The same k=1 readout design can be dropped into beam search, MCTS or weighted voting without changing the search algorithm itself.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the linear-representation premise holds more broadly, other post-generation tasks (safety checks, preference ranking, tool routing) could likewise consume the generation cache instead of re-tokenizing.
  • Architecture-sharing is currently a hard limit; a natural extension is a learned projector that maps one model's KV tensors into another's attention space.
  • The exponential decay of marginal information with readout depth suggests that multi-token verifiers will rarely justify their extra cost once a single-token adapter is well trained.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes KV-PRM, a process reward model that scores multi-agent trajectories by attending a single verify token to the generator’s pre-existing KV cache (with a LoRA adapter), reducing per-call scoring cost from O(d L^{2}) to O(d L). Under a linear-representation hypothesis and a low-rank reward assumption, it proves that the KV cache is information-theoretically richer than decoded text (Proposition 1, Theorem 1) and that a depth-1 readout already captures most extractable reward information with exponentially diminishing returns for larger k (Theorem 2). Empirically, on MATH, GSM8K, AIME 2024/2025 with Qwen3-0.6B/4B/8B under sequential and hierarchical MAS topologies, KV-PRM matches or exceeds a same-architecture Text-PRM baseline under beam search, MCTS and weighted voting, while reporting up to ~5000× fewer scoring FLOPs, 15–37× lower latency and up to 34.2× lower per-sequence memory. A short KV-Steering proof-of-concept shows that the differentiable KV scores can be used for gradient-based latent message optimization at agent handoffs.

Significance. If the accuracy–efficiency trade-off holds under broader baselines, the work removes a genuine systems bottleneck for PRM-guided multi-agent test-time search: scoring cost that currently scales with generation cost. The O(L) reduction is architecturally clean, the theory supplies a clear justification for why single-token KV readout should not lose information relative to text, and the empirical suite covers three model scales, four datasets, three search algorithms and two topologies. Explicit strengths include complete proofs under stated assumptions (Appendix A), wall-clock latency and memory measurements on GH200, and the observation that KV scores are differentiable, opening latent-space steering that text PRMs cannot do. The same-architecture constraint and Linear Representation Hypothesis are correctly listed as limitations; the contribution remains a strong systems-plus-theory result for long-horizon MAS reasoning.

major comments (2)
  1. Tables 1–3 and §5.1: the central accuracy claim (“matches or strictly outperforms text-PRMs”) is established only against a same-base-model, same-LoRA-rank, same-MCTS-label Text-PRM that re-encodes the full trajectory with the adapter active on every token. Because KV transfer requires identical architecture (§8), the comparison never tests a stronger or differently sized independent text verifier. Figure 2 (left) shows smaller Text-PRMs underperform, but does not rule out a carefully tuned same-size or larger text verifier trained with different objectives or more capacity. The “outperforms” half of the strongest claim therefore rests on a single baseline family; either add at least one stronger/independent text-PRM baseline or rephrase the claim to “matches or exceeds a same-architecture text-PRM while reducing cost by orders of magnitude.”
  2. §5.2 / Tables 1–2: accuracy numbers are reported without error bars, multiple seeds, or statistical tests. Several reported gains are small (0.1–1 pp) or mixed (e.g., GSM8K MCTS 0.6B: Text-PRM 57.77 vs KV-PRM 54.74). Without variance estimates it is hard to judge whether “matches or strictly outperforms” is robust, especially on AIME where n is small. At minimum, report standard errors over seeds or bootstrap intervals for the main tables.
minor comments (6)
  1. Assumption 1 and Theorem 2: the spectral decay rate α is free; a short sensitivity plot or a statement of the range of α consistent with the observed k-saturation in Figure 2 (right) would make the near-optimality claim more concrete.
  2. §4.1 / Eq. (8): clarify whether the LoRA adapter is applied only to the verify-token path or also modifies the cached K/V projections; the text says “adapter disabled during generation” but the precise scope of Δθ during scoring should be explicit.
  3. Figure 1 caption and cost equations: the O(d L) vs O(d L^{2}) comparison is clear, but stating the precise constants c_attn, c_ffn used for the “5000×” claim (or that it is simply L) would avoid ambiguity.
  4. §6 / Table 4: KV Steering is a nice proof-of-concept; a one-sentence note on step size, number of gradient steps T, and whether the steered cache remains valid for subsequent generation would help reproducibility.
  5. Appendix A.1–A.2: the two subsections are both titled “Proof of Theorem 1”; renumber for clarity.
  6. Related work: a brief comparison to other cache-reuse or latent-verifier ideas (beyond compression/eviction) would better situate the novelty of using KV as the reward representation.

Circularity Check

0 steps flagged

No circular derivation: theory follows from stated external assumptions plus DPI; empirics use ground-truth MCTS labels; O(L) cost is architectural, not a fitted prediction.

full rationale

Walking the load-bearing chain: (i) Proposition 1 and Theorem 1 rest on the Data Processing Inequality (Cover & Thomas) and the Linear Representation Hypothesis (Park et al.), both external; the capacity bound m′=Ω(dL/log|V|) is a counting argument under Assumption 1, not a redefinition of the target. (ii) Theorem 2 derives exponential decay of ΔIk and k=1 near-optimality from Assumption 2’s spectral decay σj=O(e−αj); the residual after k=1 is a geometric series—standard math under an explicit hypothesis, not a tautology. (iii) The O(L²)→O(L) scoring reduction is the arithmetic cost of one query token against a pre-existing cache (Eqs. 9–10), an architectural fact of the method, not a scientific prediction forced by a fit. (iv) Training labels are MCTS Q-values from exact-match to ground-truth answers; both KV-PRM and Text-PRM are trained on the same external labels, so accuracy comparisons are not self-fulfilling. Self-citations ([5],[23],[26]) appear only in related-work context and do not underwrite uniqueness or forbid alternatives. No equation equates a claimed prediction to its fitted input by construction. Score 0 is the honest outcome.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

Central claims rest on two domain assumptions (linear representation of hidden states and low-rank exponential spectral decay of reward features) plus standard information-theoretic inequalities; free parameters are mainly training hyper-parameters and the unmeasured spectral rate α; the invented entities are the KV-PRM readout itself and the optional KV-Steering procedure.

free parameters (3)
  • spectral decay rate α
    Controls the exponential bound on marginal information gain (Thm 2); never measured on real models, only assumed positive.
  • LoRA rank r=256 / α=32
    Chosen by hand for both KV-PRM and Text-PRM; affects absolute accuracy but not the relative complexity claim.
  • MCTS rollouts R=64, candidates C=4
    Label-generation hyper-parameters that determine the quality of the training signal for both scorers.
axioms (3)
  • domain assumption Linear Representation Hypothesis: hidden embeddings are linear combinations of a linearly independent semantic basis with ternary coefficients
    Assumption 1; used to prove the Ω(d/log|V|) capacity advantage of KV cache over text (Prop 1).
  • domain assumption Low-rank reward structure with exponentially decaying singular values σ_j = O(e^{-αj})
    Assumption 2; required for the exponential decay of marginal information gain and k=1 near-optimality (Thm 2).
  • standard math Data Processing Inequality applied to the Markov chain Y → H → x
    Standard information-theory fact used to show I(H;Y) ≥ I(x;Y) (proof of Thm 1).
invented entities (2)
  • KV-PRM single-token readout independent evidence
    purpose: Map pre-existing generation KV cache to a scalar process reward via one verify token and LoRA
    Core algorithmic contribution; independent evidence is the empirical match to text-PRM accuracy at far lower cost.
  • KV Steering no independent evidence
    purpose: Gradient ascent on the KV cache at agent hand-offs to raise the PRM score before the next agent generates
    Proof-of-concept enabled by differentiability of the KV readout; results are small and preliminary.

pith-pipeline@v1.1.0-grok45 · 20225 in / 2764 out tokens · 30419 ms · 2026-07-13T05:02:02.270340+00:00 · methodology

0 comments
read the original abstract

Process Reward Models (PRMs) have been proven to be highly effective in guiding test-time scaling (TTS) methods, which significantly boost the capabilities of LLM-based multi-agent systems. However, existing PRMs are text-based: they re-encode the entire trajectory text from scratch. In long multi-agent rollouts, the scoring cost, growing quadratically with respect to sequence length L, creates a severe computational bottleneck, severely limiting PRMs' application in long-context scenarios. To resolve this, we introduce KV-PRM, a highly efficient process reward model that eliminates the heavy text re-encoding by directly reading the KV cache produced naturally during the LLM's generation phase. By processing a single "verify token" against the pre-existing KV cache, KV-PRM reduces the scoring cost from O(L^2) to O(L). We formally prove that the KV cache contains strictly greater information capacity than text, and is more efficient for downstream reward modeling. Empirically, across the MATH, GSM8K, and AIME benchmarks, KV-PRM matches or strictly outperforms text-PRMs under various TTS methods such as Beam Search, MCTS, and Weighted Voting, with up to a 5,000x reduction in scoring FLOPs, a 37x reduction in latency, and a 34x reduction in per-sequence memory footprint compared to text-based PRMs.

Figures

Figures reproduced from arXiv: 2607.09153 by Haibo Jin, Haohan Wang, Kaidi Xu, Peng Kuang, Xiaopeng Yuan, Xiaoyu Han, Yanli Wang, Ye Yu.

Figure 1
Figure 1. Figure 1: Text-PRM re-encodes the full trajectory for each scoring call at [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Analysis of KV-PRM efficiency and readout depth. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Wall-clock efficiency on NVIDIA GH200 120GB GPU. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV

    cs.AI 2026-07 conditional novelty 6.5

    Source-omitted event KV rows can carry compact upstream state (semantic materialization); deliberate answer-free carriers raise recovery from 6% to 51% on Qwen3-8B, while passive natural mentions do not.

Reference graph

Works this paper leans on

35 extracted references · 19 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems (NeurIPS), 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  2. [2]

    Scaling LLM test-time compute op- timally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute op- timally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Large language model based multi-agents: A survey of progress and challenges.Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2024

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges.Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2024

    2024

  5. [5]

    Optimal aggregation of LLM and PRM signals for efficient test-time scaling

    Peng Kuang, Yanli Wang, Xiaoyu Han, Yaowenqi Liu, Kaidi Xu, and Haohan Wang. Optimal aggregation of LLM and PRM signals for efficient test-time scaling. InThe Fourteenth Inter- national Conference on Learning Representations, 2026. URL https://openreview.net/ forum?id=x85kiYqL4y

    2026

  6. [6]

    Alphazero-like tree-search can guide large language model decoding and training

    Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-like tree-search can guide large language model decoding and training. Proceedings of the International Conference on Machine Learning (ICML), 2024

    2024

  7. [7]

    Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations.arXiv preprint arXiv:2312.08935, 2024

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations.arXiv preprint arXiv:2312.08935, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    A survey of process reward models: From outcome signals to process supervisions for large language models.arXiv preprint arXiv:2510.08049, 2025

    Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, and Weinan Zhang. A survey of process reward models: From outcome signals to process supervisions for large language models.arXiv preprint arXiv:2510.08049, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    LoRA: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    MetaGPT: Meta programming for a multi-agent collaborative framework.arXiv preprint arXiv:2308.00352, 2024

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for a multi-agent collaborative framework.arXiv preprint arXiv:2308.00352, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    MASPRM: Multi-agent system process reward model.arXiv preprint arXiv:2510.24803, 2025

    Milad Yazdani, Mahdi Mostajabdaveh, Zirui Zhou, and Ying Xiong. MASPRM: Multi-agent system process reward model.arXiv preprint arXiv:2510.24803, 2025

    arXiv 2025

  13. [13]

    Bandit based Monte-Carlo planning

    Levente Kocsis and Csaba Szepesvári. Bandit based Monte-Carlo planning. InEuropean Conference on Machine Learning (ECML), pages 282–293. Springer, 2006

    2006

  14. [14]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2023

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    The linear representation hypothesis and the geometry of large language models.Proceedings of the International Conference on Machine Learning (ICML), 2024

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.Proceedings of the International Conference on Machine Learning (ICML), 2024. 10

    2024

  16. [16]

    Latent collabora- tion in multi-agent systems, 2025

    Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hang- hang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, and Ling Yang. Latent collabora- tion in multi-agent systems, 2025. URLhttps://arxiv.org/abs/2511.20639

    Pith/arXiv arXiv 2025

  17. [17]

    Theoretical guarantees for iterative alignment of self-rewarding language models, 2026

    Shi Fu, Yingjie Wang, Shengchao Hu, Peng Wang, and Dacheng Tao. Theoretical guarantees for iterative alignment of self-rewarding language models, 2026. URL https://openreview. net/forum?id=Dhd7nHdAaf

    2026

  18. [18]

    The spectral geometry of thought: Phase transitions, instruction reversal, token- level dynamics, and perfect correctness prediction in how transformers reason, 2026

    Yi Liu. The spectral geometry of thought: Phase transitions, instruction reversal, token- level dynamics, and perfect correctness prediction in how transformers reason, 2026. URL https://arxiv.org/abs/2604.15350

    Pith/arXiv arXiv 2026

  19. [19]

    An Implementation of Generative PRM, 2024

    Wei Xiong, Hanning Zhang, Nan Jiang, and Tong Zhang. An Implementation of Generative PRM, 2024. URL https://github.com/RLHFlow/RLHF-Reward-Modeling. Publication Title: GitHub repository

    2024

  20. [20]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  21. [21]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  22. [22]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Agent primitives: Reusable latent building blocks for multi-agent systems, 2026

    Haibo Jin, Kuang Peng, Ye Yu, Xiaopeng Yuan, and Haohan Wang. Agent primitives: Reusable latent building blocks for multi-agent systems, 2026. URL https://arxiv.org/abs/2602. 03695

    2026

  24. [24]

    Process reward models that think.arXiv preprint arXiv:2504.16828, 2025

    Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng, Moontae Lee, Honglak Lee, and Lu Wang. Process reward models that think.arXiv preprint arXiv:2504.16828, 2025

    arXiv 2025

  25. [25]

    GenPRM: Scaling test-time compute of process reward models via generative reasoning.arXiv preprint arXiv:2504.00891, 2025

    Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, and Bowen Zhou. GenPRM: Scaling test-time compute of process reward models via generative reasoning.arXiv preprint arXiv:2504.00891, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Tim-prm: Verifying multimodal reasoning with tool-integrated prm, 2025

    Peng Kuang, Xiangxiang Wang, Wentao Liu, Jian Dong, and Kaidi Xu. Tim-prm: Verifying multimodal reasoning with tool-integrated prm, 2025. URL https://arxiv.org/abs/2511. 22998

    2025

  27. [27]

    Improve mathematical reasoning in language models by automated process supervision.arXiv preprint arXiv:2406.06592, 2024

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi. Improve mathematical reasoning in language models by automated process supervision.arXiv preprint arXiv:2406.06592, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240, 2024

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Improv- ing factuality and reasoning in language models through multiagent debate.arXiv preprint arXiv:2305.14325, 2023

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improv- ing factuality and reasoning in language models through multiagent debate.arXiv preprint arXiv:2305.14325, 2023

    Pith/arXiv arXiv 2023

  30. [30]

    CAMEL: Communicative agents for “mind” exploration of large language model society.Advances in Neural Information Processing Systems, 2023

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: Communicative agents for “mind” exploration of large language model society.Advances in Neural Information Processing Systems, 2023

    2023

  31. [31]

    A survey on large language model acceleration based on KV cache manage- ment.arXiv preprint arXiv:2412.19442, 2024

    Haoyang Li et al. A survey on large language model acceleration based on KV cache manage- ment.arXiv preprint arXiv:2412.19442, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Pith/arXiv arXiv 2023

  33. [33]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles (SOSP), 2023. 11

    2023

  34. [34]

    Tree of thoughts: Deliberate problem solving with large language models.arXiv preprint arXiv:2305.10601, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.arXiv preprint arXiv:2305.10601, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    ?” (token ID determined by the tokenizer). The judgment tokens are “+

    Thomas M Cover and Joy A Thomas.Elements of Information Theory. Wiley-Interscience, 2nd edition, 2006. 12 A Theoretical Proofs A.1 Proof of Theorem 1 Part (a): Information sufficiency.The generation process defines the Markov chain Y→H→x : trajectory quality Y influences the generation dynamics that produce KV cacheH= (K,V) , and text x= decode(H) is a de...

    2006