Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading

Christopher G. Brinton; Dusit Niyato; Minrui Xu

REVIEW 4 major objections 5 minor 1 cited by

Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A test-time reinforcement learning policy jointly decides which LLMs to cache on edge servers and which inference requests to offload, cutting simulated system costs by at least 30% while preserving reasoning accuracy.

desk verdict A relevant edge-LLM caching/offloading idea that is undermined by a broken convergence proof and a self-referential cost model; the 30% claim does not stand as-is, but the paper deserves a serious referee. read the letter →

arxiv 2501.14205 v1 pith:HIBUTUQX submitted 2025-01-24 cs.NI

Minrui Xu , Dusit Niyato , Christopher G. Brinton This is my paper · ORCID

classification cs.NI

keywords mobileedgecomputinglargelanguagemodelsmodelcachinginferenceoffloadingtest-timetrainingdeepreinforcementlearningchain-of-thoughtreasoningdoubleDutchauction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Serving long-context LLMs at the edge is hard because, unlike stateless computation offloading, a cached model's context grows with every multi-round interaction, changing accuracy, latency, and resource use. The paper tries to establish that a joint policy for which LLMs to cache on edge GPUs and which inference requests to offload to the cloud can be learned by a test-time deep reinforcement learning algorithm, and that this policy cuts total system cost by at least 30% over baselines while keeping reasoning accuracy high. This matters because, if true, mobile edge networks could host capable LLM agents for perception and reasoning tasks without shipping every request to the cloud, adapting as user contexts evolve. The claimed savings come from a cost model that counts switching, transmission, computation, and accuracy costs under GPU-memory, energy, and context-window constraints.

What carries the argument

The machine that carries the argument is the test-time training (TTT) model used as the actor-critic network inside T2DRL: instead of a fixed LSTM hidden state, the hidden state is itself a learnable model updated by a self-supervised loss during inference, giving linear complexity and the ability to adapt to context changes in real time. Around it, the paper builds an 'age of thoughts' (AoT) state variable $\kappa^t_{n,i,m}$ that tracks how much useful reasoning context has accumulated for each cached model, and an accuracy model $A^t_{n,i,m}=\alpha_{i,m}\log(1/(\beta \kappa^t_{n,i,m}))$ that converts that state into a cost. The self-consistency chain-of-thought consensus step over $J_{i,m}$ reasoning paths is what makes the accuracy term improve with more paths, giving the policy a concrete lever beyond raw cache hits.

What would settle it

Run T2DRL and the baselines on a real edge GPU serving an open LLM with self-consistent chain-of-thought on a reasoning benchmark, measuring actual dollar or energy cost, latency, and accuracy over multi-round interactions. If measured accuracy as a function of age-of-thought does not approximately follow $\alpha \log(1/(\beta \kappa))$, or if T2DRL does not achieve at least the predicted cost reduction at equal accuracy, the paper's central claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that long-context LLM serving should be treated as a stateful, context-aware resource-allocation problem rather than a static caching or offloading problem, and that the optimal deployment strategy can be found online by a test-time deep reinforcement learning algorithm (T2DRL). The authors formulate a mixed-integer program minimizing a weighted sum of edge switching cost, transmission cost, computation cost, and accuracy cost plus cloud inference cost, with constraints on GPU memory, energy, and context-window utilization. They model the growth and decay of reasoning context through an 'age of thoughts' variable and relate it to accuracy via $A=\alpha \log(1/(\beta \kappa))$, grounded in an ambiguity-convergence analysis of chain-of-thought and self-consistent chain-of-thought prompting. In their simulated edge environment, T2DRL converges in roughly 500 episodes, reduces system cost by at least 30% compared with FIFO, LFU, and cloud-only baselines, and by about 20% compared with PPO, while improving accuracy on MultiArith, GSM8K, Commonsense QA, and related reasoning benchmarks.

Load-bearing premise

The load-bearing premise is that the analytical accuracy formula $A=\alpha \log(1/(\beta \kappa))$ with hand-set or uncalibrated coefficients faithfully represents how real LLM accuracy and serving cost respond to context age; if that curve is wrong for real models, the reported 30% cost reduction is an artifact of the simulation.

Editorial extensions

If this is right

Edge servers can serve multi-round LLM agent interactions without defaulting to the cloud, because the policy learns to keep models whose context is still productive and evict models whose context has faded.
The same policy can absorb the extra computation of self-consistent chain-of-thought: more reasoning paths raise accuracy, and T2DRL keeps total cost lower than single-path or static policies.
The double Dutch auction adds a market layer on top of the RL decisions, giving individual rationality, incentive compatibility, and budget balance while raising social welfare over an iterative double auction.
The framework generalizes the notion of edge caching from static services to stateful, context-growing workloads, so caching decisions must be made jointly with inference offloading at every time step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: the same test-time-trained actor-critic recipe could apply to other stateful edge services, such as streaming perception pipelines or multi-turn vision-language agents, wherever hidden context evolves during serving.
Editorial extension: the 30% figure is computed inside the paper's analytical cost model; a deployment study would need to calibrate $\alpha$, $\beta$, and the vanishing factor against real model accuracy before expecting these savings in practice.
Editorial extension: the auction could be run over context-window reservations or KV-cache memory rather than whole models, which would make the market finer-grained and potentially increase social welfare further.
A natural next test is to compare T2DRL against a context-window-aware LFU variant on real serving traces; if the real cost curves deviate from Eq. (14), the policy would need recalibration rather than retraining from scratch.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take: the paper has a real engineering idea, but the central theoretical result and the experimental evidence are not in a publishable state. The hook—using TTT-based actor-critic to jointly manage model caching and inference offloading for long-context LLMs at the edge—is a reasonable and timely problem. The DDA auction is clearly described and the market formulation is sensible. I'd credit the authors for identifying a gap that the stateless-offloading literature doesn't cover.

The soft spots are serious. Theorem 1 is supposed to show that SC-CoT reduces ambiguity with more reasoning paths, but the proof is incomplete: it says "combining Eqs. (14)–(18)" when no such equations exist in the proof, and the final inequality bounds the product over the length of a single path rather than the product over J_i,m paths stated in the theorem. I checked Eq. (14), and the stress-test concern holds: A_t = α log(1/(βκ_t)) decreases as the AoT κ_t grows and goes negative for κ_t > 1/β, which is the opposite of what the SC-CoT convergence result claims. Since the accuracy cost in Eq. (18) inherits this, the cost model used to obtain the 30% reduction is internally inconsistent with the paper's own theory. On top of that, the experiments are simulated entirely inside this cost model, with hand-set coefficients, no code, no error bars, and no real edge measurements. The reward is the negative of the same total cost used as the evaluation metric, so the 30% number is partly built into the setup.

I want to be fair: these are not fatal if the authors can replace Eq. (14) with a sane accuracy model and prove the theorem properly. But as it stands, the quantitative headline is not supported.

Would I cite it? No, not for the quantitative claims. Would I bring it to a reading group? Maybe, as a cautionary example of how a self-consistent simulation can still mislead. Would I send it to peer review? Yes—a serious referee should see it, because the problem matters and the flaws are fixable in principle. My expectation is major revision or rejection.

Referee Report

4 major / 5 minor

Summary. The paper proposes a joint model caching and inference offloading framework for serving long-context LLMs at mobile edge networks. It models edge servers with GPU memory and energy constraints, introduces an SC-CoT reasoning model with an 'age of thoughts' state, and formulates a mixed-integer program that is solved by a test-time reinforcement learning algorithm (T2DRL) based on TTT actor-critic networks. A double Dutch auction is added for resource allocation. The central claim is that T2DRL reduces system costs by at least 30% compared to baselines while maintaining LLM reasoning accuracy.

Significance. If the central claim were sound, the paper would address a timely problem: adaptive cache and offloading decisions for long-context LLM agents under edge resource constraints. The use of test-time training in an actor-critic architecture and the attempt to connect SC-CoT reasoning convergence to a cost model are relevant ideas. However, the claimed 30% cost reduction is evaluated entirely inside the authors' own uncalibrated analytical cost model, and the accuracy component of that model has a sign inconsistency with the paper's own convergence theory. The theoretical convergence theorem also contains a proof gap. As a result, the main quantitative claim is not currently supported.

major comments (4)

[Section III.C, Theorem 1 proof (Eqs. (5)-(11))] The proof of Theorem 1 states 'by combining Eqs. (14)-(18)' but no such equations exist at that point in the paper; the proof jumps from Eq. (10) to Eq. (11). Moreover, Eq. (11) has a product over y=1 to c_j,i on the right-hand side, whereas Theorem 1 in Eq. (5) requires a product over all J_i,m reasoning paths. This is not a cosmetic mismatch: the claimed compounding benefit of multiple SC-CoT paths is precisely what the theorem is supposed to establish, and the proof as written does not show that the product over paths emerges from the marginalization over R_j. The convergence analysis is load-bearing for the accuracy-cost model, so this gap must be fixed.
[Section III.C, Eq. (14) and Section III.D, Eq. (18)] The accuracy term A_t = alpha log(1/(beta * kappa)) in Eq. (14) decreases monotonically as kappa (the age of thoughts) increases and becomes negative for kappa > 1/beta. This is the opposite of the monotonicity claimed by Corollary 1 and Theorem 1, where longer and more numerous reasoning paths reduce ambiguity and should improve accuracy. The accuracy cost in Eq. (18) inherits this sign problem, so the edge inference cost is not a faithful representation of the model's own performance theory. Since the 30% cost-reduction claim is computed inside this cost model, the sign error invalidates the central experimental claim regardless of coefficient calibration.
[Section IV.A.c, reward function and Section VI, Fig. 3(a)] The reward function is defined as the negative of the total system cost L_total, and the experimental metric in Fig. 3(a) is the very same L_total from Eq. (21). The T2DRL algorithm is therefore optimizing the exact quantity on which the 'at least 30% reduction' claim is evaluated. This circularity alone would not be fatal if the cost model were validated against real measurements, but no such validation is provided; all cost coefficients are hand-set (Section VI, first paragraph), and the accuracy term is the analytically specified Eq. (14). The claimed cost reduction is thus partly built into the evaluation metric.
[Section VI, Figs. 3-5 and Tables I-II] The experiments are entirely simulated under the authors' cost model; no real edge LLM serving measurements of cost, latency, or energy are presented. Tables I and II are static accuracy tables from the original SC-CoT paper [14], not measurements from the proposed framework. The claim in the abstract that T2DRL 'guarantees the performance of LLM agents in real-world perception and reasoning tasks' is therefore not supported by any real-world deployment or trace-driven evaluation.

minor comments (5)

[Section III.B, Eq. (4) and Theorem 1] The notation in Lemma 1 and Theorem 1 is inconsistent: the product index in Eq. (4) uses y over c_i steps, while Theorem 1 in Eq. (5) uses a product over j=1 to J_i,m but the factors are written as epsilon(E_i,j)/(1-epsilon(E_i,j)); the relationship between the step-level product and the path-level product should be clarified.
[Section III.D, Eq. (16)] Equation (16) writes the transmission cost as R_t_{n,i,m} l_n,com d_i (1-b_t_{n,i,m}) but the text introduces 'l_n^{edge}' as the unit transmission cost; the notation l_n,com is undefined and appears to be a typo. Similarly, the superscripts on l_n in Eqs. (15)-(19) are inconsistent.
[Section V, 'Property Analysis'] The claims of individual rationality, incentive compatibility, and strong budget balance for the DDA are stated without formal proofs. Since the auction is a secondary contribution, the statements should at least be accompanied by a sketch or a reference to a standard double-auction result.
[Section VI, Figs. 3-5] The figures show aggregated cost and accuracy curves but no error bars or confidence intervals, and the number of independent runs besides the seed 42 is not reported. The paper also does not state whether the accuracy values in Fig. 5 are obtained by running the actual LLMs or are read off from the static tables in Tables I and II.
[Eq. (13) and Fig. 5(b)] The vanishing factor Delta_t_{i,m} reduces kappa_{n,i,m} in Eq. (13), and Eq. (14) makes accuracy an increasing function of kappa. Yet Fig. 5(b) reports that reasoning accuracy improves as the vanishing factor increases from 1 to 4. This is internally inconsistent with the model equations unless an additional mechanism is intended, which the text does not explain.

Circularity Check

3 steps flagged · score 6.0 of 10

The headline 30% cost reduction is computed inside the same cost model that defines the RL reward, while the theoretical accuracy term rests on a same-author citation and a self-referential proof.

self citation load bearing [Section III.C, Lemma 1 and Theorem 1 (Eqs. 4-5); reference [32]]
"The proof of Lemma 1 can be found in [32] and this lemma essentially supports the concept of CoT by establishing a mathematical relationship that quantifies how the ambiguity in input influences the accuracy of the CoT reasoning process."

Theorem 1 and Corollary 1, which constitute the paper's claimed performance-convergence analysis, are built directly on Lemma 1. The proof of Lemma 1 is not given here; it is deferred to [32], whose first two authors (M. Xu and D. Niyato) are also the first two authors of this paper. This is not an external, machine-checked, or parameter-free independent result, and it is the load-bearing support for the accuracy term A_t (Eq. 14) that enters the cost model, the reward, and the reported cost saving.
fitted input called prediction [Section IV.A.c (Reward Function) and Section VI (Fig. 3(a))]
"The reward at time t, denoted as rt n, is defined as the negative of the total system cost, ensuring that lower costs yield higher rewards as rt n = −L n total(a t n, b t n). ... Compared to baseline algorithms such as FIFO, LFU, and Cloud, T2DRL achieves at least a 30% reduction in system costs."

The experimental metric for the headline claim is the same scalar L_total that the T2DRL policy is trained to maximize through reward r = −L_total. Every component of L_total (switching, transmission, computation, accuracy, cloud) is defined in Section III.D with hand-set coefficients, and no real edge-serving measurements are used to calibrate or test it. Thus the reported 'at least 30% reduction' is a statement about how well the policy optimizes its own training objective in the simulated cost model, not an independent prediction; the evaluation loop is closed by construction.

1 more flagged steps

other [Section III.B-C, Eq. (14) and Theorem 1 proof (Eq. 11)]
"To reflect the SC-CoT process, we modify the performance Eq. (5) to include the most consistent answer selection mechanism as A_t = α log(1/(βκ_t)). ... Finally, by combining Eqs. (14)-(18), we can get |p_m(D_{i,j}|d_{i,0}, E_{i,j}) − q̂(D_{i,j}|d_{i,0}, c*)| ≤ η ..."

Eq. (14) is presented as a modification of Eq. (5), but Eq. (5) is the bound stated in Theorem 1. The proof of Theorem 1 then says 'by combining Eqs. (14)-(18)'—i.e., it invokes the later cost equations, including Eq. (14) that is itself justified by the theorem. The derivation chain is therefore self-referential: the theorem supports the accuracy formula, and the theorem's proof appeals to that formula. This is an omitted/defective proof rather than an external derivation.

full rationale

The paper is not wholly circular: the benchmark accuracies in Tables I and II are external few-shot performance numbers, and the DDA social-welfare analysis is a self-contained mechanism comparison. However, the central cost claim is evaluated inside the same cost model used as the RL reward, so the 30% reduction is partly built into the evaluation. The theoretical accuracy term that feeds the cost model is imported from a same-author citation ([32]) for Lemma 1, and the proof of Theorem 1 references the later cost equations, creating a self-referential derivation chain. These issues do not make every contribution vacuous, but they substantially weaken the independence of the headline cost-reduction claim; the result is best read as an in-model optimization demonstration rather than an externally validated prediction.

Assumptions & free parameters 7 free parameters · 7 assumptions · 1 invented entities

The central claim rests on an uncalibrated cost model with several hand-set parameters, plus a convergence bound delegated to a same-author prior paper. The free parameters listed above directly shape the cost function that T2DRL optimizes and that the experiments use as the evaluation metric.

free parameters (7)

reasoning gain factor beta
Appears in Eq. (14) inside log(1/(beta * kappa)); no calibration or value is given, and it controls the accuracy cost and the RL reward.
consensus factor zeta_i
Appears in Eq. (13) and controls the accumulation of age-of-thoughts; no empirical basis is provided.
vanishing factor Delta^t_{i,m} = varied 1 to 4 in Figs. 4(d) and 5(b)
Appears in Eq. (13) and models context relevance decay; chosen for sensitivity analysis without external grounding.
zero-shot accuracy alpha_{i,m} = from LaMDA-137B and PaLM-540B tables
Taken from the published tables in [14] and used in Eq. (14) to define accuracy cost.
switching cost coefficient lambda = 1e-5
Hand-set in Section VI; it scales the switching cost in Eq. (15).
edge and cloud transmission cost coefficients = 0.0001 and 0.0075
Hand-set in Section VI; they determine the transmission cost terms in Eqs. (16) and (20).
accuracy cost coefficient = 2.5
Hand-set in Section VI; it scales the accuracy cost term even though Eq. (18) does not display it explicitly.

assumptions (7)

domain assumption True contexts c* are uniformly distributed.
Assumption 1 in Section III.B; needed to set the skewness parameter gamma_n(c*) = 1 in the Lemma 1 proof.
domain assumption Ambiguity epsilon(E_i) is bounded by sigma in [0, 1/2].
Assumption 2 in Section III.B; used to bound the products in Lemma 1 and Theorem 1.
domain assumption Ambiguity tends to zero as sequence length goes to infinity.
Assumption 3 in Section III.B; drives Corollary 1 and the length threshold k*_{i,sigma}.
ad hoc to paper Lemma 1 from [32] is correct and applicable.
The proof of Lemma 1 is not in this paper; [32] is a same-first-author prior work, making Theorem 1 depend on an unverified self-cited result.
ad hoc to paper Accuracy follows A = alpha * log(1/(beta * kappa)) in Eq. (14).
No derivation from Theorem 1 or from empirical data is provided; this function determines the accuracy cost and the RL reward.
domain assumption LLMs are universal density approximators as used in Definition 1 from [29].
Background assumption for the approximation p_m approximately q_hat used in the ambiguity derivations.
ad hoc to paper The analytical cost model in Eqs. (15)-(20) matches real LLM serving costs.
The 30% reduction claim is computed under this model; no measurements from actual edge LLM serving are provided.

invented entities (1)

Age of Thoughts (AoT) kappa_{n,i,m}
purpose: Quantifies accumulated reasoning context and its decay, feeding into the accuracy cost and the RL reward.
Introduced via Eqs. (12)-(13) with consensus and vanishing factors; no empirical validation is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading." pith.science (2026). https://pith.science/paper/HIBUTUQX

@misc{pith2026250114205,
  author       = {Pith},
  title        = {Pith review of: Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIBUTUQX}},
  note         = {Machine review of arXiv:2501.14205}
}

read the original abstract

Large Language Models (LLMs) can perform zero-shot learning on unseen tasks and few-shot learning on complex reasoning tasks. However, resource-limited mobile edge networks struggle to support long-context LLM serving for LLM agents during multi-round interactions with users. Unlike stateless computation offloading and static service offloading in edge computing, optimizing LLM serving at edge servers is challenging because LLMs continuously learn from context which raises accuracy, latency, and resource consumption dynamics. In this paper, we propose a joint model caching and inference offloading framework that utilizes test-time deep reinforcement learning (T2DRL) to optimize deployment and execution strategies for long-context LLM serving. In this framework, we analyze the performance convergence and design an optimization problem considering the utilization of context windows in LLMs. Furthermore, the T2DRL algorithm can learn in both the training phase and the testing phase to proactively manage cached models and service requests and adapt to context changes and usage patterns during execution. To further enhance resource allocation efficiency, we propose a double Dutch auction (DDA) mechanism, which dynamically matches supply and demand while maximizing social welfare. Finally, experimental results demonstrate that the T2DRL algorithm can reduce system costs by at least 30% compared to baselines while guaranteeing the performance of LLM agents in real-world perception and reasoning tasks.

Figures

Figures reproduced from arXiv: 2501.14205 by the authors.

**Figure 2.** The T2DRL algorithm utilizes the test-time training [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Convergence analysis of the proposed T2DRL algorithm. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Performance comparison of the proposed T2DRL algorithm under different environment settings. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

**Figure 5.** Figure 5: Reasoning accuracy of the T2DRL algorithm under different (a) reasoning paths; (b) vanishing factor. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: Social welfare verse market size. relevance decays over time, as it adapts to mitigate information loss and prioritize critical reasoning components. Across both subfigures, T2DRL achieves high accuracy consistently, with MultiArith and ARC emerging as the top-performi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters
cs.DC 2026-07 conditional novelty 6.0 of 10

A per-query orchestrator that jointly picks LLM family, size, quantization, and edge device via fitted predictors and a BILP-based heuristic reduces measured latency and keeps accuracy stable versus two simpler baselines.

Reference graph

Works this paper leans on

40 extracted references · 16 canonical work pages · cited by 1 Pith paper

[14]

Self-consistency improves chain of thought reasoning in language models,

X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint arXiv:2203.11171 , 2022

arXiv 2022
[1]

When large language model agents meet 6g networks: Percep- tion, grounding, and alignment,

M. Xu, N. Dusit, J. Kang, Z. Xiong, S. Mao, Z. Han, D. I. Kim, and K. B. Letaief, “When large language model agents meet 6g networks: Percep- tion, grounding, and alignment,” arXiv preprint arXiv:2401.07764, 2024

arXiv 2024
[2]

Language mod- els are few-shot learners,

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

1901
[3]

Sparks of artificial general intelligence: Early experiments with gpt-4,

S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Ka- mar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg et al. , “Sparks of artificial general intelligence: Early experiments with gpt-4,” arXiv preprint arXiv:2303.12712, 2023

arXiv 2023
[4]

Large language model (llm) for telecommu- nications: A comprehensive survey on principles, key techniques, and opportunities,

H. Zhou, C. Hu, Y . Yuan, Y . Cui, Y . Jin, C. Chen, H. Wu, D. Yuan, L. Jiang, D. Wu et al. , “Large language model (llm) for telecommu- nications: A comprehensive survey on principles, key techniques, and opportunities,” arXiv preprint arXiv:2405.10825 , 2024

arXiv 2024
[5]

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving

P. Zeng, Z. Ning, J. Zhao, W. Cui, M. Xu, L. Guo, X. Chen, and Y . Shan, “The cap principle for llm serving,” arXiv preprint arXiv:2405.11299 , 2024

work page Pith review arXiv 2024
[6]

Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference,

S. Ye, J. Du, L. Zeng, W. Ou, X. Chu, Y . Lu, and X. Chen, “Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference,” arXiv preprint arXiv:2405.17245 , 2024

arXiv 2024
[7]

Retention-aware container caching for serverless edge computing,

L. Pan, L. Wang, S. Chen, and F. Liu, “Retention-aware container caching for serverless edge computing,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications . IEEE, 2022, pp. 1069– 1078

work page 2022

Show all 40 references

[8]

Cache- enabled federated learning systems,

Y . Liu, L. Su, C. Joe-Wong, S. Ioannidis, E. Yeh, and M. Siew, “Cache- enabled federated learning systems,” in Proceedings of the Twenty- fourth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, 2023, pp. 1–11

2023
[9]

Edgeadaptor: Online configuration adaption, model selection and re- source provisioning for edge dnn inference serving at scale,

K. Zhao, Z. Zhou, X. Chen, R. Zhou, X. Zhang, S. Yu, and D. Wu, “Edgeadaptor: Online configuration adaption, model selection and re- source provisioning for edge dnn inference serving at scale,” IEEE Transactions on Mobile Computing , vol. 22, no. 10, pp. 5870–5886, 2022

2022
[10]

Cooperative service caching and workload scheduling in mobile edge computing,

X. Ma, A. Zhou, S. Zhang, and S. Wang, “Cooperative service caching and workload scheduling in mobile edge computing,” in IEEE INFO- COM 2020-IEEE Conference on Computer Communications . IEEE, 2020, pp. 2076–2085

2020
[11]

Memserve: Context caching for disaggregated llm serving with elastic memory pool,

C. Hu, H. Huang, J. Hu, J. Xu, X. Chen, T. Xie, C. Wang, S. Wang, Y . Bao, N. Sun et al. , “Memserve: Context caching for disaggregated llm serving with elastic memory pool,”arXiv preprint arXiv:2406.17565, 2024

2024 arXiv
[12]

Llm-dcache: Improving tool- augmented llms with gpt-driven localized data caching,

S. Singh, M. Fore, A. Karatzas, C. Lee, Y . Jian, L. Shangguan, F. Yu, I. Anagnostopoulos, and D. Stamoulis, “Llm-dcache: Improving tool- augmented llms with gpt-driven localized data caching,” arXiv preprint arXiv:2406.06799, 2024

2024 arXiv
[13]

Chain-of-thought prompting elicits reasoning in large language models,

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

2022
[15]

Prox- imal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[16]

Learning to (learn at test time): Rnns with expressive hidden states,

Y . Sun, X. Li, K. Dalal, J. Xu, A. Vikram, G. Zhang, Y . Dubois, X. Chen, X. Wang, S. Koyejo et al., “Learning to (learn at test time): Rnns with expressive hidden states,” arXiv preprint arXiv:2407.04620 , 2024

2024 arXiv
[17]

Pushing large language models to the 6g edge: Vision, challenges, and opportunities,

Z. Lin, G. Qu, Q. Chen, X. Chen, Z. Chen, and K. Huang, “Pushing large language models to the 6g edge: Vision, challenges, and opportunities,” arXiv preprint arXiv:2309.16739 , 2023

2023 arXiv
[18]

Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services,

Z. Yang, Y . Yang, C. Zhao, Q. Guo, W. He, and W. Ji, “Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services,” arXiv preprint arXiv:2405.14636 , 2024

2024 arXiv
[19]

Titanic: Towards production federated learning with large language models,

N. Su, C. Hu, B. Li, and B. Li, “Titanic: Towards production federated learning with large language models,” in IEEE INFOCOM, 2024

2024
[20]

Generative inference of large language models in edge computing: An energy efficient approach,

X. Yuan, H. Li, K. Ota, and M. Dong, “Generative inference of large language models in edge computing: An energy efficient approach,” in 2024 International Wireless Communications and Mobile Computing (IWCMC). IEEE, 2024, pp. 244–249

2024
[21]

Two time-scale joint service caching and task offloading for uav-assisted mobile edge computing,

R. Zhou, X. Wu, H. Tan, and R. Zhang, “Two time-scale joint service caching and task offloading for uav-assisted mobile edge computing,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications. IEEE, 2022, pp. 1189–1198

2022
[22]

Trim- caching: Parameter-sharing edge caching for ai model downloading,

G. Qu, Z. Lin, Q. Chen, J. Li, F. Liu, X. Chen, and K. Huang, “Trim- caching: Parameter-sharing edge caching for ai model downloading,” arXiv preprint arXiv:2404.14204 , 2024

2024 arXiv
[23]

A3c-based computation offloading and service caching in cloud-edge computing networks,

Z. Wang, M. Li, L. Zhao, H. Zhou, and N. Wang, “A3c-based computation offloading and service caching in cloud-edge computing networks,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) . IEEE, 2022, pp. 1–2

2022
[24]

Deepcache: A deep learning based framework for content caching,

A. Narayanan, S. Verma, E. Ramadan, P. Babaie, and Z.-L. Zhang, “Deepcache: A deep learning based framework for content caching,” in Proceedings of the 2018 Workshop on Network Meets AI & ML , 2018, pp. 48–53

2018
[25]

Deep reinforcement learning-based computation offloading and distributed edge service caching for mobile edge computing,

M. Xie, J. Ye, G. Zhang, and X. Ni, “Deep reinforcement learning-based computation offloading and distributed edge service caching for mobile edge computing,” Computer Networks, p. 110564, 2024

2024
[26]

Neighboring- aware caching in heterogeneous edge networks by actor-attention-critic learning,

Y . Zhao, R. Li, C. Wang, X. Wang, and V . C. Leung, “Neighboring- aware caching in heterogeneous edge networks by actor-attention-critic learning,” in ICC 2021-IEEE International Conference on Communica- tions. IEEE, 2021, pp. 1–6. 14

2021
[27]

Cooperative task offloading and service caching for digital twin edge networks: A graph attention multi- agent reinforcement learning approach,

Z. Yao, S. Xia, Y . Li, and G. Wu, “Cooperative task offloading and service caching for digital twin edge networks: A graph attention multi- agent reinforcement learning approach,” IEEE Journal on Selected Areas in Communications, 2023

2023
[28]

Large language models (llms) inference offloading and resource allocation in cloud-edge com- puting: An active inference approach,

Y . He, J. Fang, F. R. Yu, and V . C. Leung, “Large language models (llms) inference offloading and resource allocation in cloud-edge com- puting: An active inference approach,” IEEE Transactions on Mobile Computing, 2024

2024
[29]

Are transformers universal approximators of sequence-to-sequence func- tions?

C. Yun, S. Bhojanapalli, A. S. Rawat, S. Reddi, and S. Kumar, “Are transformers universal approximators of sequence-to-sequence func- tions?” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=ByxRM0Ntvr

2020
[30]

Why can large language models generate correct chain-of-thoughts?

R. Tutunov, A. Grosnit, J. Ziomek, J. Wang, and H. Bou-Ammar, “Why can large language models generate correct chain-of-thoughts?” arXiv preprint arXiv:2310.13571, 2023

2023 arXiv
[31]

A latent space theory for emergent abilities in large language models,

H. Jiang, “A latent space theory for emergent abilities in large language models,” arXiv preprint arXiv:2304.09960 , 2023

2023 arXiv
[32]

Cached model-as-a-resource: Provisioning large language model agents for edge intelligence in space-air-ground integrated networks,

M. Xu, D. Niyato, H. Zhang, J. Kang, Z. Xiong, S. Mao, and Z. Han, “Cached model-as-a-resource: Provisioning large language model agents for edge intelligence in space-air-ground integrated networks,” arXiv preprint arXiv:2403.05826, 2024

2024 arXiv
[33]

Imagebind: One embedding space to bind them all,

R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V . Alwala, A. Joulin, and I. Misra, “Imagebind: One embedding space to bind them all,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15 180–15 190

2023
[34]

Solving general arithmetic word problems,

S. Roy and D. Roth, “Solving general arithmetic word problems,” arXiv preprint arXiv:1608.01413, 2016

2016 arXiv
[35]

Think you have solved question answering? try arc, the ai2 reasoning challenge,

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv preprint arXiv:1803.05457 , 2018

2018 arXiv
[36]

Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies,

M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and J. Berant, “Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 346–361, 2021

2021
[37]

Commonsenseqa: A question answering challenge targeting commonsense knowledge,

A. Talmor, J. Herzig, N. Lourie, and J. Berant, “Commonsenseqa: A question answering challenge targeting commonsense knowledge,”arXiv preprint arXiv:1811.00937, 2018

2018 arXiv
[38]

Adversarial glue: A multi-task benchmark for robustness evaluation of language models,

B. Wang, C. Xu, S. Wang, Z. Gan, Y . Cheng, J. Gao, A. H. Awadallah, and B. Li, “Adversarial glue: A multi-task benchmark for robustness evaluation of language models,” arXiv preprint arXiv:2111.02840, 2021

2021 arXiv
[39]

Improve diverse text generation by self labeling conditional variational auto encoder,

Y . Zhang, Y . Wang, L. Zhang, Z. Zhang, and K. Gai, “Improve diverse text generation by self labeling conditional variational auto encoder,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2019, pp. 2767–2771

2019
[40]

Training verifiers to solve math word problems,

K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al., “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 , 2021

2021 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [14]

Self-consistency improves chain of thought reasoning in language models,

X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint arXiv:2203.11171 , 2022

arXiv 2022

[2] [1]

When large language model agents meet 6g networks: Percep- tion, grounding, and alignment,

M. Xu, N. Dusit, J. Kang, Z. Xiong, S. Mao, Z. Han, D. I. Kim, and K. B. Letaief, “When large language model agents meet 6g networks: Percep- tion, grounding, and alignment,” arXiv preprint arXiv:2401.07764, 2024

arXiv 2024

[3] [2]

Language mod- els are few-shot learners,

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

1901

[4] [3]

Sparks of artificial general intelligence: Early experiments with gpt-4,

S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Ka- mar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg et al. , “Sparks of artificial general intelligence: Early experiments with gpt-4,” arXiv preprint arXiv:2303.12712, 2023

arXiv 2023

[5] [4]

Large language model (llm) for telecommu- nications: A comprehensive survey on principles, key techniques, and opportunities,

H. Zhou, C. Hu, Y . Yuan, Y . Cui, Y . Jin, C. Chen, H. Wu, D. Yuan, L. Jiang, D. Wu et al. , “Large language model (llm) for telecommu- nications: A comprehensive survey on principles, key techniques, and opportunities,” arXiv preprint arXiv:2405.10825 , 2024

arXiv 2024

[6] [5]

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving

P. Zeng, Z. Ning, J. Zhao, W. Cui, M. Xu, L. Guo, X. Chen, and Y . Shan, “The cap principle for llm serving,” arXiv preprint arXiv:2405.11299 , 2024

work page Pith review arXiv 2024

[7] [6]

Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference,

S. Ye, J. Du, L. Zeng, W. Ou, X. Chu, Y . Lu, and X. Chen, “Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference,” arXiv preprint arXiv:2405.17245 , 2024

arXiv 2024

[8] [7]

Retention-aware container caching for serverless edge computing,

L. Pan, L. Wang, S. Chen, and F. Liu, “Retention-aware container caching for serverless edge computing,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications . IEEE, 2022, pp. 1069– 1078

work page 2022