arxiv: 2604.17353 · v1 · submitted 2026-04-19 · 💻 cs.AI · cs.DC

Recognition: unknown

Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling

Runlin Guo, Yansong Xu, Youwei Xiao, Yuhao Luo, Yun Liang, Zizhang Luo

Authors on Pith no claims yet

Pith reviewed 2026-05-10 06:28 UTC · model grok-4.3

classification 💻 cs.AI cs.DC

keywords multi-agent systemslarge language modelsinference optimizationlogits cacheagent-aware schedulingtask decompositionscaling algorithms

0 comments

The pith

Hive enables efficient scaling of multi-agent LLM systems by caching logits across reasoning paths and scheduling resources according to agent contributions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Hive as a multi-agent infrastructure to support scaling at both algorithm and task levels in large language model deployments. At the algorithm level, it addresses redundancy in inference-time computations across multiple reasoning branches by reusing logits. At the task level, it allows task decomposition across agents with scheduling that accounts for each agent's role. If successful, this would allow more complex agentic workflows without proportional increases in compute cost or resource waste.

Core claim

Hive provides a description frontend to specify per-agent behaviors and test-time scaling algorithms, paired with a backend that includes a Logits Cache for reusing intermediate logits across redundant sampling paths and an Agent-Aware Scheduling mechanism that allocates compute and KV-cache resources based on quantified agent contributions.

What carries the argument

The Logits Cache, which reuses logits from overlapping sampling paths to reduce redundancy, and Agent-Aware Scheduling, which quantifies and uses agent contributions for resource allocation.

Load-bearing premise

That logits from overlapping sampling paths can be safely reused without altering the final model outputs and that agent contributions can be quantified precisely enough to guide effective scheduling.

What would settle it

Running an experiment where logits are cached and reused but the final outputs differ from a no-cache baseline, or where the scheduling leads to higher miss rates than standard methods.

Figures

Figures reproduced from arXiv: 2604.17353 by Runlin Guo, Yansong Xu, Youwei Xiao, Yuhao Luo, Yun Liang, Zizhang Luo.

**Figure 1.** Figure 1: Case study on a Hardware Verification Agent can plan over multiple steps, invoke external tools, and coordinate toward complex goals[2, 23, 26]. In these emerging workloads, inference is no longer a collection of independent prompt-response pairs; instead, it becomes a stateful process in which context accumulates across turns, intermediate results are repeatedly reused, and multiple reasoning branches o… view at source ↗

**Figure 2.** Figure 2: (a) An example of branching reasoning in Tree-of-Thought. (b) A detailed view of the sampling process for State 4. (c) Tree-of-Thoughts process using Logits Cache 2 Background 2.1 Large Language Model (LLM) Inference Modern LLMs[19, 29, 31] are typically served through an autoregressive inference process. Given an input prompt, the serving system first executes a prefill stage to process all prompt tokens … view at source ↗

**Figure 3.** Figure 3: Simplified R3A[21] multi-agent workflow example with five agents: Decision, Patcher, Viewer, Summary, and Searcher. 2.3 Multi-Agent System A multi-agent system (MAS) consists of multiple interacting agents that collaborate through communication, information sharing, and decentralized decision-making to accomplish a common objective. MAS has been widely proposed as a pivotal pathway for harnessing collect… view at source ↗

**Figure 4.** Figure 4: Profiling results of the R3A [21] multi-agent system based on 105 captured calls. Bars: Token usage across agent roles, including non-cached input tokens, output tokens, and cached prefix tokens. Line: Invocation counts across agent roles. 3 Motivation 3.1 Cross-Path Redundancy At the algorithm level, we observed that test-time scaling, especially hybrid TTS methods, introduces substantial computational r… view at source ↗

**Figure 5.** Figure 5: Overview of Hive Multi-Agent Infrastructure determined solely by generic access order. As a result, KV cache from frequently reused agents such as Programmer or Reviewer may be evicted simply because they are less recent than the newly inserted Search states, even though they are far more likely to be reused in subsequent execution. Such agent-agnostic eviction can significantly degrade overall efficiency … view at source ↗

**Figure 6.** Figure 6: Hive Front End: the Python-based pseudo-code to describe agents and supervisors, the generated flow graph, and the runtime system. with the decorator agent. The bug_repair agent cooperates with the context_gather agent to help gather the related code snippets and fix the bug. The bug_repair agent first spawns the context_gather agent. Then, in each loop, it receives the gathered code from the context_gathe… view at source ↗

**Figure 7.** Figure 7: Profiles for logit cache on R3A [21]. For token overlap, we fix top-p=1 and leave top-k unconstrained, thereby excluding additional sampling truncation effects from the analysis. For logit uncertainty: (1) Normalized entropy of the token distribution at each decode step. (2) Normalized importance score used by hotspot selection. (3) Step-level variability statistics, including entropy, top-1 probability, s… view at source ↗

**Figure 8.** Figure 8: Performance of Logits Cache under different temperatures for two replay policies. (a)–(d) show the results for step-wise sampling, and (e)–(h) show the results for hotspot sampling. The reported metrics include mean TPS, P50 TPS, speedup, and logits-cache hit rate. receive a higher score if it is invoked more frequently, achieves better cache efficiency, exhibits stronger concurrency. Hive models this uti… view at source ↗

**Figure 9.** Figure 9: Trajectory of agent priority scores over 21 rounds. For each round, scores are normalized across all agents. workloads to measure how effectively it reduces replay overhead during repeated resampling. For task-level scaling, we evaluate the Multi-Agent Scheduler on multi-agent workloads to study whether agent-aware resource allocation improves execution efficiency over existing scheduling policies. Toget… view at source ↗

read the original abstract

Large language models are increasingly deployed as complex agentic systems that scale with task complexity. While prior work has extensively explored model- and system-level scaling, algorithm- and task-level scaling remain largely unaddressed, constraining the full potential of agentic systems. At the algorithm level, allocating additional inference-time computation can enhance workflow capacity but introduces cross-path redundancy: overlapping computations across multiple reasoning branches. At the task level, complex tasks can be decomposed into subproblems and delegated across multiple agents for improved scalability and parallelism. However, existing infrastructures' scheduling is unaware of the existence of multiple agents, missing opportunities to optimize resource allocation. We propose Hive, a multi-agent infrastructure that enables algorithm- and task-level scaling. Hive features a description frontend that captures per-agent behavior and supports test-time scaling algorithms. Leveraging this specification, our backend introduces two key mechanisms: Logits Cache that reuses intermediate logits across redundant sampling paths to mitigate cross-path redundancy at the algorithm level, and Agent-Aware Scheduling that efficiently allocates compute and KV-cache resources according to agent contributions at the task level. Experiments show that Logits Cache achieves an average speedup of $1.11\times$-$1.76\times$ for re-sampling, and Agent-Aware Scheduling reduces the hotspot miss rate by $33\%$-$51\%$.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Hive proposes a practical infrastructure for multi-agent LLM scaling with logits caching and agent-aware scheduling, but the speedup claims rest on unverified output equivalence.

read the letter

The main thing to know about this paper is that Hive adds a frontend for describing agent behaviors and a backend with two mechanisms: a logits cache to reuse computations across overlapping sampling paths, and agent-aware scheduling to allocate resources based on per-agent contributions. It reports 1.11-1.76x speedups on re-sampling and 33-51% drops in hotspot misses, targeting gaps in algorithm- and task-level scaling that model- or system-level work has left open. That combination is new enough to be worth noting for people building complex agent workflows. The paper does a decent job framing the redundancy problem in multi-branch reasoning and the limitations of generic schedulers in multi-agent setups, then offering concrete infrastructure pieces to address them. If the cache works without side effects, it could be a useful efficiency lever for test-time scaling. The soft spots are concentrated in the experimental side. The abstract states performance numbers but supplies no baselines, methodology, error bars, or description of how prefix identity is detected and logits are re-injected. The stress-test concern is on target: without explicit checks that cached paths produce identical token sequences and downstream outputs, the wall-clock gains cannot be trusted as correctness-preserving. If temperature, randomness, or KV-cache state is altered even slightly, the reported benefits become meaningless. The full paper might contain those details, but nothing in the provided material confirms equivalence or rules out divergence. This work is for researchers and engineers focused on LLM serving infrastructure and multi-agent systems who need practical ways to reduce waste in parallel sampling or shared-resource allocation. Readers already working on agentic pipelines could extract usable ideas even if they have to re-implement and validate the components. It deserves a serious referee because the problem framing is timely and the proposed mechanisms are specific, though any review would need to press hard on verification of the cache and the experimental protocol.

Referee Report

3 major / 2 minor

Summary. The paper proposes Hive, a multi-agent infrastructure for algorithm- and task-level scaling of LLM-based agentic systems. It introduces a description frontend to capture per-agent behavior and support test-time scaling, plus a backend with two mechanisms: Logits Cache, which reuses intermediate logits across redundant sampling paths to reduce cross-path computation, and Agent-Aware Scheduling, which allocates compute and KV-cache resources according to quantified agent contributions. The central empirical claims are that Logits Cache delivers 1.11×–1.76× average speedup on re-sampling and Agent-Aware Scheduling reduces hotspot miss rate by 33%–51%.

Significance. If the logits-reuse mechanism is shown to preserve output equivalence and the scheduling gains are reproducible with proper controls, the work could offer a practical infrastructure layer that improves efficiency for complex multi-agent workflows, addressing redundancy in inference-time scaling and agent-aware resource allocation where prior systems fall short.

major comments (3)

[Logits Cache implementation] The Logits Cache mechanism (described in the backend section) claims to mitigate cross-path redundancy by reusing logits from overlapping sampling paths, yet provides no specification of prefix-identity detection, logits re-injection into the sampler, temperature/random-state handling, or any equivalence test confirming that cached paths produce identical token sequences and downstream agent outputs to the uncached baseline.
[Experiments / Results] The experimental results reporting 1.11×–1.76× speedup for Logits Cache and 33%–51% hotspot-miss-rate reduction for Agent-Aware Scheduling contain no details on models, tasks, number of trials, baselines, statistical measures, or error bars, rendering it impossible to assess whether the numbers support the scaling claims.
[Agent-Aware Scheduling] Agent-Aware Scheduling relies on accurate quantification of per-agent contributions for resource allocation, but the manuscript does not describe the quantification method, its sensitivity to task decomposition choices, or ablation against standard schedulers, which is load-bearing for the reported miss-rate improvement.

minor comments (2)

[Abstract] The abstract states performance ranges without any experimental context; adding a brief clause on evaluation scale would improve clarity.
[Notation and figures] Ensure consistent terminology for 'KV-cache' and 'logits' across sections and figures.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We address each major comment below and will revise the paper to incorporate the requested clarifications and additional details.

read point-by-point responses

Referee: [Logits Cache implementation] The Logits Cache mechanism (described in the backend section) claims to mitigate cross-path redundancy by reusing logits from overlapping sampling paths, yet provides no specification of prefix-identity detection, logits re-injection into the sampler, temperature/random-state handling, or any equivalence test confirming that cached paths produce identical token sequences and downstream agent outputs to the uncached baseline.

Authors: We agree that the current description is high-level and lacks these critical implementation specifics. In the revised manuscript, we will expand the backend section with a detailed account of prefix-identity detection via KV-cache prefix matching, the logits re-injection logic into the sampler, explicit handling of temperature and random states to preserve determinism, and results from equivalence tests confirming identical token sequences and downstream agent outputs between cached and baseline paths. revision: yes
Referee: [Experiments / Results] The experimental results reporting 1.11×–1.76× speedup for Logits Cache and 33%–51% hotspot-miss-rate reduction for Agent-Aware Scheduling contain no details on models, tasks, number of trials, baselines, statistical measures, or error bars, rendering it impossible to assess whether the numbers support the scaling claims.

Authors: We acknowledge that the experimental reporting is insufficient for full reproducibility and evaluation. The revised version will include comprehensive details on the models evaluated, the specific tasks and benchmarks used, the number of trials per configuration, the baselines employed, and statistical measures including standard deviations and error bars to support the reported speedups and miss-rate reductions. revision: yes
Referee: [Agent-Aware Scheduling] Agent-Aware Scheduling relies on accurate quantification of per-agent contributions for resource allocation, but the manuscript does not describe the quantification method, its sensitivity to task decomposition choices, or ablation against standard schedulers, which is load-bearing for the reported miss-rate improvement.

Authors: We recognize that the quantification method and supporting analyses are not adequately described. In the revision, we will add a detailed explanation of the per-agent contribution quantification (combining static task analysis and dynamic profiling), sensitivity analysis to variations in task decomposition, and ablations against standard schedulers such as FIFO and shortest-job-first to substantiate the hotspot miss-rate improvements. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical infrastructure paper with no derivations or fitted predictions

full rationale

The paper describes a new multi-agent infrastructure (Hive) featuring Logits Cache and Agent-Aware Scheduling, then reports empirical speedups (1.11×–1.76×) and miss-rate reductions (33%–51%) from experiments. No mathematical derivation chain, first-principles predictions, parameter fitting, or self-citation load-bearing steps exist. Claims rest on direct measurements of implemented components rather than any reduction of outputs to inputs by construction. This is self-contained engineering work; the reader's circularity score of 1.0 aligns with the absence of any load-bearing circular pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 3 invented entities

The paper introduces new software mechanisms and relies on standard assumptions about LLM inference and multi-agent task decomposition without introducing fitted parameters or new physical entities.

axioms (2)

domain assumption Intermediate logits from LLM inference can be cached and reused across multiple reasoning paths without affecting the final output quality.
This underpins the Logits Cache mechanism described in the abstract.
domain assumption Agent contributions to task completion can be quantified to guide resource allocation in scheduling.
Basis for Agent-Aware Scheduling.

invented entities (3)

Hive infrastructure no independent evidence
purpose: To enable algorithm- and task-level scaling in multi-agent LLM systems.
Core new system proposed.
Logits Cache no independent evidence
purpose: Reuses intermediate logits to mitigate cross-path redundancy.
New mechanism for algorithm-level scaling.
Agent-Aware Scheduling no independent evidence
purpose: Allocates compute and KV-cache based on agent contributions.
New mechanism for task-level scaling.

pith-pipeline@v0.9.0 · 5547 in / 1455 out tokens · 53344 ms · 2026-05-10T06:28:04.909205+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages · 9 internal anchors

[1]

Gulavani, Alexey Tumanov, and Ramachandran Ramjee

Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. arXiv:2403.02310 [cs.LG]https://arxiv.org/abs/ 2403.02310

work page arXiv 2024
[2]

anomalyco. 2026. anomalyco/opencode: The open source AI coding agent.https://github.com/anomalyco/opencode. GitHub repository, accessed 2026-03-24

2026
[3]

Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang
[4]

Forest-of-thought: Scaling test-time compute for enhancing llm reasoning.arXiv preprint arXiv:2412.09078(2024)

work page arXiv 2024
[5]

Zhuohang Bian, Feiyang Wu, Teng Ma, and Youwei Zhuo. 2025. Token- cake: A KV-Cache-centric Serving Framework for LLM-based Multi- Agent Applications.arXiv preprint arXiv:2510.18586(2025)

work page arXiv 2025
[6]

Hao Mark Chen, Zhiwen Mo, Guanxi Lu, Shuang Liang, Lingxiao Ma, Wayne Luk, and Hongxiang Fan. 2026. FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoning. InProceedings of the 31st ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems, Volume 2(USA)(ASPLOS ’26). Association for Computing Machinery...

work page doi:10.1145/3779212.3790161 2026
[7]

Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. 2025. EfficientQAT: Efficient Quantization-Aware Training for Large Language Models. InProceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vienna, Austria, 10081–1...

2025
[8]

Renze Chen, Zhuofeng Wang, Beiquan Cao, Tong Wu, Size Zheng, Xiuhong Li, Xuechao Wei, Shengen Yan, Meng Li, and Yun Liang. 2024. ArkVale: Efficient Generative LLM Inference with Recallable Key- Value Eviction. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2024).https://doi.org/10.52202/079017-3595

work page doi:10.52202/079017-3595 2024
[9]

Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Jun- hua Zhao, and Xiuqiang He. 2024. Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects. arXiv:2401.03428 [cs.AI]https://arxiv.org/abs/2401.03428

work page arXiv 2024
[10]

Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo. 2024. Cost- Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention. arXiv:2403.19708 [cs.CL]https://arxiv.org/ abs/2403.19708

work page arXiv 2024
[11]

Training Compute-Optimal Large Language Models

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Henni- gan, Eric Noland, Katie Millican, George van den Driessche, Bog- dan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and L. Sifre....

work page internal anchor Pith review arXiv 2022
[12]

Junyan Hu, Parijat Bhowmick, Inmo Jang, Farshad Arvin, and Alexan- der Lanzon. 2021. A Decentralized Cluster Formation Containment Framework for Multirobot Systems.IEEE Transactions on Robotics37, 6 (2021), 1936–1955.https://doi.org/10.1109/TRO.2021.3071615

work page doi:10.1109/tro.2021.3071615 2021
[13]

Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, Huizi Mao, Yujun Lin, Hanrong Ye, Sifei Liu, Ka Chun Cheung, Hongxu Yin, et al. 2025. QeRL: Beyond Efficiency–Quantization-enhanced Reinforcement Learning for LLMs.arXiv preprint arXiv:2510.11696(2025)

work page arXiv 2025
[14]

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He
[15]

Deepspeed ulysses: System optimizations for enabling train- ing of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509(2023)

work page internal anchor Pith review arXiv 2023
[16]

Hao Kang, Ziyang Li, Xinyu Yang, Weili Xu, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora
[17]

ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System.arXiv preprint arXiv:2602.13692(2026)

work page arXiv 2026
[18]

Scaling Laws for Neural Language Models

Jared Kaplan, Sam McCandlish, Thomas Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models.ArXiv abs/2001.08361 (2020).https://api.semanticscholar.org/CorpusID: 210861095

work page internal anchor Pith review Pith/arXiv arXiv 2020
[19]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica
[20]

InProceedings of the 29th symposium on operating systems principles

Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th symposium on operating systems principles. 611–626
[21]

Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, and Dayi- heng Liu. 2025. Start: Self-taught reasoner with tools. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing. 13523–13564

2025
[22]

Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges.Vicinagearth1, 1 (2024), 9.https://doi.org/10.1007/s44336- 024-00009-2

work page doi:10.1007/s44336- 2024
[23]

Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. 2025. Deepseek-v3. 2: Pushing the frontier of open large language models.arXiv preprint arXiv:2512.02556(2025)

work page internal anchor Pith review arXiv 2025
[24]

Hao Liu, Matei Zaharia, and Pieter Abbeel. 2023. Ring attention with blockwise transformers for near-infinite context.arXiv preprint arXiv:2310.01889(2023). 12 Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling

work page internal anchor Pith review arXiv 2023
[25]

Zizhang Luo, Fan Cui, Kexing Zhou, Runlin Guo, Mile Xia, Hongyuan Hou, and Yun Liang. 2025. R3A: Reliable RTL Repair Framework with Multi-Agent Fault Localization and Stochastic Tree-of-Thoughts Patch Generation. arXiv:2511.20090 [cs.AR]https://arxiv.org/abs/2511.20090

work page arXiv 2025
[26]

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Kather- ine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark
[27]

InThirty- seventh Conference on Neural Information Processing Systems.https: //openreview.net/forum?id=S37hOerQLB

Self-Refine: Iterative Refinement with Self-Feedback. InThirty- seventh Conference on Neural Information Processing Systems.https: //openreview.net/forum?id=S37hOerQLB
[28]

Moonshot AI. 2026. Kimi Code: Next-Gen AI Code Agent | Automated Programming & CLI.https://www.kimi.com/codeAccessed: 2026-04- 15

2026
[29]

nlile. 2026. 24-game.https://huggingface.co/datasets/nlile/24-game Accessed: 2026-04-11

2026
[30]

OpenAI. 2025. Introducing GPT-5.4.https://openai.com/index/ introducing-gpt-5-4/. Accessed: 2026-04-15

2025
[31]

openclaw. 2026. openclaw/openclaw: Your own personal AI assistant. Any OS. Any Platform. The lobster way.https://github.com/openclaw/ openclaw. GitHub repository, accessed 2026-03-24

2026
[32]

Zaifeng Pan, Ajjkumar Patel, Zhengding Hu, Yipeng Shen, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, and Yufei Ding. 2025. KVFlow: Efficient prefix caching for accelerating LLM-based multi-agent work- flows.arXiv preprint arXiv:2507.07400(2025)

work page arXiv 2025
[33]

Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, and Ricardo Bianchini. 2024. Splitwise: Efficient generative llm inference using phase splitting. In2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 118–132

2024
[34]

Qwen Team. [n. d.].Qwen3-Coder-Next Technical Report. Technical Re- port.https://github.com/QwenLM/Qwen3-Coder/blob/main/qwen3_ coder_next_tech_report.pdfAccessed: 2026-02-03

2026
[35]

Benedek Rozemberczki, Lauren Watson, Péter Bayer, Hao-Tsung Yang, Olivér Kiss, Sebastian Nilsson, and Rik Sarkar. 2022. The Shapley Value in Machine Learning. arXiv:2202.05594 [cs.LG]https://arxiv. org/abs/2202.05594

work page arXiv 2022
[36]

Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. 2026. Kimi K2. 5: Visual Agentic Intelligence.arXiv preprint arXiv:2602.02276(2026)

work page internal anchor Pith review arXiv 2026
[37]

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. At- tention Is All You Need.CoRRabs/1706.03762 (2017). arXiv:1706.03762 http://arxiv.org/abs/1706.03762

work page internal anchor Pith review Pith/arXiv arXiv 2017
[38]

Junlin Wang, Jue WANG, Ben Athiwaratkun, Ce Zhang, and James Zou
[39]

InThe Thirteenth International Conference on Learning Representations

Mixture-of-Agents Enhances Large Language Model Capabilities. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id=h0ZfDIrj7T
[40]

Yongtong Wu, Shaoyuan Chen, Yinmin Zhong, Rilin Huang, Yixuan Tan, Wentao Zhang, Liyue Zhang, Shangyan Zhou, Yuxuan Liu, Shun- feng Zhou, Mingxing Zhang, Xin Jin, and Panpan Huang. 2026. Du- alPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference. arXiv:2602.21548 [cs.DC]https://arxiv.org/abs/2602.21548

work page arXiv 2026
[41]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

work page internal anchor Pith review Pith/arXiv arXiv 2025
[42]

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems36 (2023), 11809–11822

2023
[43]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations

2022
[44]

Jintao Zhang, Jia Wei, Pengle Zhang, Xiaoming Xu, Haofeng Huang, Haoxu Wang, Kai Jiang, Jianfei Chen, and Jun Zhu. 2025. Sageatten- tion3: Microscaling fp4 attention for inference and an exploration of 8-bit training.arXiv preprint arXiv:2505.11594(2025)

work page arXiv 2025
[45]

Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muen- nighoff, et al . 2025. A survey on test-time scaling in large lan- guage models: What, how, where, and how well?arXiv preprint arXiv:2503.24235(2025)

work page internal anchor Pith review arXiv 2025
[46]

Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. 2024. Sglang: Efficient execution of structured lan- guage model programs.Advances in neural information processing systems37 (2024), 62557–62583

2024
[47]

Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xu- anzhe Liu, Xin Jin, and Hao Zhang. 2024. {DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 193–210. 13

2024