Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges

Haozhao Wang; Jingling Yuan; Ruixuan Li; Rui Zhang; Senyao Li; Song Guo; Tianwei Zhang; Wenchao Xu; Xian Zhong

REVIEW 5 major objections 4 minor 4 cited by

Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey establishes a unified taxonomy of edge-cloud LLM-SLM collaboration, covering inference-time assignment, division, and mixture plus five training-time paradigms.

desk verdict A useful but over-claimed survey of edge-cloud LLM-SLM collaboration; the taxonomy is reasonable, but the 'first systematic foundation' claim is not supported by the paper's own cited prior surveys. read the letter →

arxiv 2507.16731 v1 pith:L2AXAXIY submitted 2025-07-22 cs.DC

Senyao Li , Haozhao Wang , Wenchao Xu , Rui Zhang , Song Guo , Jingling Yuan , Xian Zhong , Tianwei Zhang

show 1 more author

Ruixuan Li

This is my paper

classification cs.DC

keywords edge-cloudcollaborationsmalllanguagemodels(SLMs)large(LLMs)collaborativeinferencespeculativedecodingknowledgedistillationtaskroutingfederatedlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that deploying language models is no longer a choice between cloud-only LLMs and compressed edge-only SLMs: the viable path is a collaborative paradigm in which both cooperate, and that paradigm has a describable shape. The survey's central deliverable is a unified taxonomy that sorts inference-time collaboration into task assignment, task division, and mixture strategies at task and token granularity, and training-time collaboration into five paradigms spanning distillation, parameter fusion, adapters, SLM-driven supervision, and cloud-guided capability injection. A reader should care because the taxonomy turns a scattered collection of routing, offloading, speculative-decoding, and distillation papers into a design space that engineers can navigate when building low-latency, privacy-preserving, personalized edge-cloud AI systems. The paper also positions itself as the first systematic review that covers both inference and training phases together, and it closes by identifying open challenges such as evidence-based uncertainty estimation.

What carries the argument

The object that carries the argument is the taxonomy itself, a two-axis classification of collaboration strategies between cloud LLMs and edge SLMs. On the inference axis, the categories are task assignment (whole-request routing), task division (routing, computation offloading, and early exit), and mixture strategies at task granularity (orchestration, historical enhancement, retrieval-augmented generation) and at token granularity (draft-verify speculative decoding, self-speculative decoding, skeleton completion, and token-tree verification). On the training axis, the categories are distillation-based collaboration, multi-SLM parameter fusion, adapter-based modular training, SLM-driven LLM supervision, and cloud-guided capability injection. The taxonomy works as a coordinate system: any existing or proposed collaboration scheme can be located by its inference category, its training category, and its granularity, which is what lets the survey compare approaches that earlier reviews treated as unrelated.

What would settle it

Collect a fresh sample of edge-cloud LLM-SLM collaboration papers published in the last two years and attempt to classify each into the taxonomy's three inference categories (task assignment, task division, mixture at task or token level) and five training paradigms; if a substantial fraction fits none of the categories, or a published system coordinates inference and training in a way the taxonomy cannot describe, the systematic-foundation claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that LLM-SLM collaboration across the edge-cloud continuum is not an ad hoc bag of techniques but a structured design space with two axes. Along the inference axis, collaboration proceeds by task assignment (route the whole request to the SLM or the LLM), task division (route, offload, or early-exit to split execution), or mixture strategies that combine assignment and division at the task or token level, with token-level mixture encompassing speculative decoding where the edge SLM drafts and the cloud LLM verifies. Along the training axis, collaboration proceeds through distillation-based transfer, multi-SLM parameter fusion, adapter-based modular tuning, SLM-driven supervision of the LLM, and cloud-guided capability injection into the SLM. The paper's claim is that jointly covering both axes, and organizing them under one taxonomy, constitutes the first systematic foundation for the field, grounding future system and algorithm co-design.

Load-bearing premise

The survey's claim to be the first systematic foundation rests on its literature coverage being complete enough; the paper provides no search protocol or inclusion criteria, so any substantial body of omitted edge-cloud LLM-SLM work could break the taxonomy's completeness.

Editorial extensions

If this is right

System builders can use the taxonomy as a menu: to cut latency, choose token-level mixture such as speculative decoding; to protect privacy, choose task assignment that keeps data local and only escalates uncertain requests.
Designers can now combine an inference strategy with a training strategy deliberately, for instance pairing edge-drafted speculative decoding with bidirectional distillation rather than treating the two as separate research areas.
Evaluation of collaborative systems should follow the benchmarks the survey collects, using user- or device-level non-IID partitions, since centralized IID benchmarks misrepresent edge deployment.
The open-challenges section points to a concrete technical direction: replacing softmax-confidence uncertainty with evidence-based (unnormalized logit) uncertainty would make early-exit and fallback decisions more reliable in open-ended generation.
Privacy-preserving collaboration is a stated requirement in the taxonomy's framing, with local adaptation, federated updates, and secure retrieval identified as the mechanisms that keep sensitive data on edge devices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the taxonomy's two axes are the right coordinates, then the field's maturity could be measured by how many real systems occupy both axes at once; a natural extension is a benchmark suite that scores systems on inference granularity and training collaboration independently.
Editorial inference: the survey's call for evidence-based uncertainty suggests a falsifiable prediction, namely that routers using Dirichlet-style evidential uncertainty will outperform softmax-confidence routers specifically on open-ended generation tasks where multiple continuations are valid; this is a testable extension the paper does not run.
Editorial inference: the taxonomy implies a gray zone of hybrid granularities, such as systems that switch between task-level and token-level cooperation mid-request based on communication budget; the paper does not quantify when each granularity wins, leaving an empirical comparison as a natural next step.
Editorial inference: the paper's own completeness claim is structurally dependent on its corpus selection, so an independent audit that classifies a fresh sample of recent edge-cloud LLM-SLM papers into the taxonomy's categories would directly test whether the systematic-foundation framing holds.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a readable, well-organized survey with a plausible taxonomy for edge-cloud LLM-SLM collaboration, and it covers a lot of ground: roughly 295 references, mostly recent, organized along two axes—inference (task assignment, task division, mixture at task/token granularity) and training (five collaborative paradigms). The tables and figures are a decent starting map for someone new to the area, and the inclusion of industrial frameworks (Walle, Luoxi) is a nice touch. As a survey, it is genuinely useful.

The main problem is the novelty claim. The abstract and Section 1.1 call this the 'first systematic foundation' and the 'first comprehensive review' jointly covering inference and training. But the paper itself cites Chen et al. [162], which covers pipelining, routing, distillation, and fusion for LLM-SLM collaboration, and Niu et al. [163], which reviews collaborative learning between device-side small models and cloud-based large models from system, algorithmic, and application perspectives. Table 1 asserts differences (e.g., [162] 'lacks a unified algorithmic design space for inference', [163] 'does not cover inference across modalities/tasks') but provides no category-by-category comparison that would let a reader verify those gaps. Without that, the central 'firstness' claim is asserted, not demonstrated. The stress-test note is right on this point.

There are also concrete editorial problems. References [34] and [84] are the same FS-GEN paper; [12] is the SpecExec paper but is cited in text as 'Hao et al.'; Section 2.2.4 has a dangling heading followed immediately by Section 2.3; and Section 1.3.3 contains uncited quantitative claims (90% accuracy in customer service, 20% LoRA gain, 40% efficiency gains). For a survey whose value depends on reliability, these need to be fixed.

The absence of a search protocol or inclusion criteria compounds the coverage concern. The corpus may be comprehensive, but the reader cannot check, and the 'systematic' claim is weakened.

None of this is fatal. The survey does not need to be 'first' to be useful; the taxonomy is a reasonable organizing device, and the paper is a solid snapshot of the design space. Who should read it: graduate students and researchers entering the area, and system or algorithm folks who want a quick map of the field.

Recommendation: send it to peer review, but with an explicit request to (1) add a search protocol or at least describe selection criteria, (2) tone down the firstness claim and replace it with a concrete comparison to [162] and [163], and (3) fix the citation and editing errors. As it stands, it is a conditional reject; after those revisions, it could be a dependable reference survey.

Referee Report

5 major / 4 minor

Summary. This survey proposes a unified taxonomy of cloud-edge collaboration between small language models (SLMs) on edge devices and large language models (LLMs) in the cloud. For inference, it categorizes methods into task assignment, task division, and mixture-based collaboration at task and token granularity. For training, it identifies five collaborative paradigms including distillation, parameter fusion, adapters, SLM-driven supervision, and cloud-guided capability injection. It also reviews datasets, benchmarks, deployment frameworks, privacy-preserving methods, and open challenges, and claims to be the first systematic foundation for LLM-SLM collaboration across both inference and training.

Significance. If the claimed novelty and comprehensiveness hold, the survey would provide a valuable organizing framework for a rapidly growing research area. Its strengths include a broad corpus of recent work (2023 onward), a clear two-axis structure (inference vs. training), comparative tables (e.g., Tables 2, 4, 6), and concrete industrial examples. The discussion of evidence-based uncertainty estimation in the conclusion is a useful forward-looking research direction. However, the central 'first systematic foundation' claim is not yet substantiated against the prior surveys the authors themselves cite, and several internal errors currently reduce the paper's reliability as a reference work.

major comments (5)

[§1.1 and Table 1] The abstract and Section 1.1 claim that this is the 'first systematic foundation' and the 'first comprehensive review' jointly covering inference and training for LLM-SLM collaboration. However, the survey itself cites Chen et al. [162] and Niu et al. [163] as prior surveys of LLM-SLM collaboration. Table 1 differentiates this survey from [162] and [163] only via one-line glosses (e.g., 'lacks a unified algorithmic design space for inference', 'does not cover inference across modalities/tasks') without a category-by-category comparison. If either prior survey already contains a comparable taxonomy, the novelty claim is false. Because the survey's contribution is organizational, this is a load-bearing point. Please (a) provide a systematic comparison mapping the proposed inference categories (task assignment, task division, mixture) and the five training paradigms onto [162] and [163], indicating which categories are truly absent, and (b) state the search protocol (databases, keywords, time window, inclusion/exclusion criteria) so that the 'systematic' claim is auditable.
[Reference list and §2.2.2] Multiple citation errors and duplicate references undermine the survey's reliability as a reference work. In §2.2.2, the text says 'Hao et al. [12, 124]', but reference [12] is the SpecExec paper (Svirschevski et al.), not by Hao; reference [124] is Hao Luo et al. Also, §2.4.1 cites [32, 40, 208] as 'early works' introducing speculative decoding, but [40] ('Guiding reasoning in small language models with llm assistance') is not a speculative decoding paper. The reference list contains duplicate entries: [34] and [84] are the same paper; [12] and [42] are the same paper; [81] and [233] are the same paper (MiniLLM); [157] and [234] are the same paper (TinyBERT). As a survey, readers depend on correct citations to locate the described techniques; these errors must be systematically corrected.
[§2.2.4] Section 2.2.4, titled 'Communication Optimization.', has no content under it: the heading is immediately followed by Section 2.3. The paragraph ending the previous page ('Efficient communication is critical...', discussing Hu et al. [267], LLMCascades [111], EdgeShard [117], PipeEdge [116], etc.) is clearly the intended material for this section, but it appears before the heading. Since Table 4 lists 'Communication Optimization § 2.2.4' as one of the four task-division paradigms, the taxonomy is incomplete as presented. Please move the paragraph under the heading and ensure each entry in Table 4 has a corresponding subsection.
[§3 and Table 6] The taxonomy of collaborative training is internally inconsistent. The Section 3 introduction states the five paradigms are 'distillation-based collaboration, adapter-based modular training, bidirectional learning, SLM-driven supervision, and cloud-guided capability injection.' Table 6 lists a different set: 'Distillation-Based Collaboration', 'Multi-SLM Parameter Fusion', 'Adapter-Based Modular Training', 'SLM-Driven LLM Supervision', and 'Cloud-Guided Capability Injection' — with 'bidirectional learning' replaced by 'Multi-SLM Parameter Fusion'. The section headings also do not cleanly map onto either list: §3.5 is titled 'Bidirectional Collaborative Learning', while §3.3 'Parameter Compatibility and Model Convergence' aligns with 'Multi-SLM Parameter Fusion'. This inconsistency means the reader cannot determine the intended set of training paradigms, which is a central contribution of the paper. Please reconcile the text, Table 6, and the section organization.
[§1.3.3] Section 1.3.3 contains quantitative claims without adequate support: '90% accuracy in customer service', 'TinyBERT achieving 96% of BERT's performance [157]', 'LoRA boosting medical QA accuracy by 20%', and '40% efficiency gains in applications like in-vehicle voice systems (200ms latency)'. The citation [157] is to the TinyBERT paper, but the specific figures (96%, 20%, 40%, 200ms) are not traced to any source and appear to be illustrative. As a survey intended to be a 'systematic foundation', these numbers should be either removed or replaced with precise claims drawn from the cited literature, with page or section references where possible. Presenting unverifiable numbers in the introduction weakens the paper's credibility.

minor comments (4)

[§1.3.3] The text says 'trillion-parameter models (e.g., LLaMA-3 405B)', but LLaMA-3 405B has 405 billion parameters, not a trillion. Please correct the descriptor or choose a genuinely trillion-parameter example.
[§5.2 and Table 8] BioMistral [225] is described as a healthcare vertical application of 'edge-cloud LLM-SLM collaborations', but the cited reference presents BioMistral as a single biomedical LLM for on-premise/hybrid deployment, not an edge-cloud collaborative system. Either explain how BioMistral fits the collaborative paradigm or remove it from this table.
[§1.1 and reference list] The text states 'we ground our discussion in recent literature published since 2023', but the reference list contains older works such as [125] from 2018 and [157] (TinyBERT, 2020). Please clarify the coverage window or adjust the statement.
[§6] The 'Future Prospects' paragraph introduces evidence-based uncertainty estimation as a recommendation, but it is not connected to the surveyed methods (e.g., which reviewed uncertainty-based routing or early-exit methods would be affected). Please add a brief link to the surveyed literature or explicitly label this as a research agenda.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey organizes existing work under a proposed taxonomy and makes no predictions or fitted-parameter claims, so there is no derivation chain that reduces to its own inputs.

full rationale

This paper is a literature survey. It contains no mathematical derivations, no fitted parameters, and no empirically validated predictions whose success could be forced by construction. The central contribution is organizational: a taxonomy of inference-time (task assignment, task division, mixture) and training-time (distillation, parameter fusion, adapters, SLM-driven supervision, cloud-guided capability injection) collaboration strategies, applied to a corpus of cited papers. The taxonomy is the paper's own construction, but applying a taxonomy to surveyed papers is not circular; the surveyed papers' reported results are external to the taxonomy and are not generated by it. The only self-citation found is reference [246], whose first author is also an author of this survey; it is cited in Section 1.1 and Table 1 as an example of system-level optimization focused surveys. That citation is descriptive background and is not load-bearing for the paper's central claim of providing a unified systematic foundation. The claim of being the 'first systematic foundation' is a novelty assertion dependent on the comparison with prior surveys such as [162] and [163] in Section 1.1 and Table 1; whether that comparison is fully demonstrated is a coverage or correctness question, not a circularity question. No quoted equation or construction shows a stated result being equivalent, by definition or by fit, to the paper's own inputs. Accordingly, no circular steps are identified, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

As a survey, the paper introduces no free parameters or invented entities. It does rely on framing assumptions: that its paper selection is comprehensive, that its taxonomy is a valid partition of the design space, and that several uncited quantitative claims are correct.

assumptions (3)

domain assumption The surveyed corpus is representative and comprehensive without a documented search protocol.
The survey's claim to be the 'first comprehensive review' depends on complete coverage of relevant work; this is assumed rather than demonstrated in the absence of a methods section.
ad hoc to paper The taxonomy's categories (assignment, division, mixture; five training paradigms) are exhaustive and non-overlapping enough to organize the literature.
The boundaries are defined by the authors; e.g., speculative decoding is discussed under token-level mixture but also relates to offloading and early exit, making the partition a modeling choice.
ad hoc to paper Quantitative claims in Section 1.3.3 are accurate.
Numbers like '90% accuracy in customer service' and '40% efficiency gains' are asserted without citations, so the survey relies on them as unverified domain assumptions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges." pith.science (2026). https://pith.science/paper/L2AXAXIY

@misc{pith2026250716731,
  author       = {Pith},
  title        = {Pith review of: Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L2AXAXIY}},
  note         = {Machine review of arXiv:2507.16731}
}

read the original abstract

As large language models (LLMs) evolve, deploying them solely in the cloud or compressing them for edge devices has become inadequate due to concerns about latency, privacy, cost, and personalization. This survey explores a collaborative paradigm in which cloud-based LLMs and edge-deployed small language models (SLMs) cooperate across both inference and training. We present a unified taxonomy of edge-cloud collaboration strategies. For inference, we categorize approaches into task assignment, task division, and mixture-based collaboration at both task and token granularity, encompassing adaptive scheduling, resource-aware offloading, speculative decoding, and modular routing. For training, we review distributed adaptation techniques, including parameter alignment, pruning, bidirectional distillation, and small-model-guided optimization. We further summarize datasets, benchmarks, and deployment cases, and highlight privacy-preserving methods and vertical applications. This survey provides the first systematic foundation for LLM-SLM collaboration, bridging system and algorithm co-design to enable efficient, scalable, and trustworthy edge-cloud intelligence.

Figures

Figures reproduced from arXiv: 2507.16731 by the authors.

**Figure 2.** Overview of collaborative paradigms between edge-cloud large and SLMs, structured along two axes: inference [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Comparison of task assignment in edge-cloud systems: (a) Agents decompose user instructions to multi-stage plans [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Illustration of three execution-phase task-division strategies: (a) request routing, dispatching queries to optimal LLM [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

**Figure 5.** Figure 5: Mixture strategies for edge-cloud model collaboration: (a) task-level, edge servers decide to respond locally, query the [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 6.** Figure 6: Hybrid SLM-LLM collaboration in RAG: the SLM encodes queries and retrieves documents via approximate nearest [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 7.** Figure 7: (a) Vanilla: the cloud verifies edge-generated drafts and rejects or regenerates invalid tokens; (b) Semantic Skeleton: [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: Model compression techniques for efficient cloud-edge collaborative training: (a) quantization of weights and [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Collaborative training with bidirectional knowledge transfer: (a) the SLM acquires task knowledge through domain [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
cs.CL 2026-07 conditional novelty 6.0 of 10

PyroDash lets a small model decide mid-generation to hand off once to a frozen LLM, achieving 64.04% accuracy (+6.36 points vs LLM) at lambda=0.05 and 96.4% lower estimated cost at lambda=0.6.
Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models
cs.CR 2026-07 conditional novelty 5.0 of 10

An edge-cloud LLM serving scheme splits the final projection along the hidden dimension, keeps cache control and LoRA modules on-device, and reports 29-46% lower per-token latency and 56-67% lower downlink payloads th...
Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training
cs.LG 2025-09 conditional novelty 4.0 of 10

Reinforcement learning post-training gives on-device LLMs an internal 'call for help' action, eliminating external routers and achieving strong math accuracy under cloud-use budgets.
Toward Edge General Intelligence with Agentic AI and Agentification: Concepts, Technologies, and Future Directions
cs.NI 2025-08 conditional novelty 4.0 of 10

A survey that organizes agentic AI for 6G edge networks into four pillars, compactness, efficiency, knowledge and reasoning, and migration, and illustrates them with prior case studies.

Reference graph

Works this paper leans on

288 extracted references · 22 canonical work pages · cited by 4 Pith papers

[162]

A survey on collaborative mechanisms between large and small language models

Yi Chen, JiaHao Zhao, and HaoHao Han. A survey on collaborative mechanisms between large and small language models. arXiv preprint arXiv:2505.07460, 2025

arXiv 2025
[163]

Collaborative learning of on-device small model and cloud-based large model: Advances and future directions

Chaoyue Niu, Yucheng Ding, Junhui Lu, et al. Collaborative learning of on-device small model and cloud-based large model: Advances and future directions. arXiv preprint arXiv:2504.15300, 2025

arXiv 2025
[84]

Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding

Kaiyan Zhang, Jianyu Wang, Ning Ding, et al. Fast and slow generating: An empirical study on large and small language models collaborative decoding. arXiv preprint arXiv:2406.12295, 2024

work page Pith review arXiv 2024
[12]

Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices

Ruslan Svirschevski, Avner May, Zhuoming Chen, et al. Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices. arXiv preprint arXiv:2406.02532, 2024

arXiv 2024
[124]

Cloud-edge collaborative intelligent inference based on distributed neural networks in power distribution networks

Hao Luo, Hui Tian, Peng Zhang, et al. Cloud-edge collaborative intelligent inference based on distributed neural networks in power distribution networks. In Int. Conf. Space-Air-Ground Comput., SAGC, pages 129–136, 2021

2021
[40]

Guiding reasoning in small language models with llm assistance

Yujin Kim, Euiin Yi, Minu Kim, et al. Guiding reasoning in small language models with llm assistance. arXiv preprint arXiv:2504.09923, 2025

arXiv 2025
[42]

Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices

Ruslan Svirschevski, Avner May, Zhuoming Chen, et al. Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024
[233]

Minillm: Knowledge distillation of large language models

Yuxian Gu, Li Dong, Furu Wei, et al. Minillm: Knowledge distillation of large language models. In Proc. 12th Int. Conf. Learn. Represent., ICLR, 2024

2024
[157]

Tinybert: Distilling BERT for natural language understanding

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al. Tinybert: Distilling BERT for natural language understanding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 4163–4174, 2020

2020
[234]

TinyBERT: Distilling BERT for natural language understanding

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al. TinyBERT: Distilling BERT for natural language understanding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 4163–4174, 2020

2020
[267]

Hybrid edge-ai framework for intelligent mobile applications: Leveraging large language models for on-device contextual assistance and code-aware automation

Liao Hu. Hybrid edge-ai framework for intelligent mobile applications: Leveraging large language models for on-device contextual assistance and code-aware automation. J. Ind. Eng. Appl. Sci. , 3(3):10–22, 2025

2025
[111]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, et al. Large language model cascades with mixture of thought representations for cost-efficient reasoning. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[117]

Edgeshard: Efficient llm inference via collaborative edge computing

Mingjin Zhang, Xiaoming Shen, Jiannong Cao, et al. Edgeshard: Efficient llm inference via collaborative edge computing. IEEE Internet Things J., 12(10):13119–13131, 2025

2025
[116]

Pipeedge: Pipeline parallelism for large-scale model inference on heterogeneous edge devices

Yang Hu, Connor Imes, Xuanang Zhao, et al. Pipeedge: Pipeline parallelism for large-scale model inference on heterogeneous edge devices. In 25th Euromicro Conf. Digit. Syst. Des., DSD , pages 298–307, 2022

2022

Show all 288 references

[1]

Online security-aware and reliability-guaranteed ai service chains provisioning in edge intelligence cloud

Yu Qiu, Junbin Liang, Victor CM Leung, et al. Online security-aware and reliability-guaranteed ai service chains provisioning in edge intelligence cloud. IEEE Trans. Mob. Comput., 2023

2023
[2]

Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness

Othmane Friha, Mohamed Amine Ferrag, Burak Kantarci, et al. Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness. IEEE Open J. Commun. Soc. , 2024

2024
[3]

Privatelora for efficient privacy preserving llm

Yiming Wang, Yu Lin, Xiaodong Zeng, et al. Privatelora for efficient privacy preserving llm. arXiv preprint arXiv:2311.14030, 2023

2023 arXiv
[4]

Mobile edge intelligence for large language models: A contemporary survey.IEEE Commun

Guanqiao Qu, Qiyuan Chen, Wei Wei, et al. Mobile edge intelligence for large language models: A contemporary survey.IEEE Commun. Surv. Tutor., 2025

2025
[5]

Multi-tier multi-node scheduling of LLM for collaborative AI computing

Mulei Ma, Chenyu Gong, Liekang Zeng, et al. Multi-tier multi-node scheduling of LLM for collaborative AI computing. In Proc. IEEE INFOCOM, pages 1–10, 2025

2025
[6]

Cambricon-llm: A chiplet-based hybrid architecture for on-device inference of 70b LLM

Zhongkai Yu, Shengwen Liang, Tianyun Ma, et al. Cambricon-llm: A chiplet-based hybrid architecture for on-device inference of 70b LLM. In Proc. 57th IEEE/ACM Int. Symp. Microarchitecture, MICRO , pages 1474–1488, 2024

2024
[7]

Hybrid sd: Edge-cloud collaborative inference for stable diffusion models

Chenqian Yan, Songwei Liu, Hongjian Liu, et al. Hybrid sd: Edge-cloud collaborative inference for stable diffusion models. arXiv preprint arXiv:2408.06646, 2024

2024 arXiv
[8]

Hybrid retrieval-augmented generation for real-time composition assistance

Xuchao Zhang, Menglin Xia, Camille Couturier, et al. Hybrid retrieval-augmented generation for real-time composition assistance. arXiv preprint arXiv:2308.04215, 2023

2023 arXiv
[9]

Dc-ccl: Device-cloud collaborative controlled learning for large vision models

Yucheng Ding, Chaoyue Niu, Fan Wu, et al. Dc-ccl: Device-cloud collaborative controlled learning for large vision models. arXiv preprint arXiv:2303.10361, 2023

2023 arXiv
[10]

Large language models (llms) inference offloading and resource allocation in cloud-edge networks: An active inference approach

Jingcheng Fang, Ying He, F Richard Yu, et al. Large language models (llms) inference offloading and resource allocation in cloud-edge networks: An active inference approach. In Proc. 98th IEEE Veh. Technol. Conf., VTC2023-Fall , pages 1–5. IEEE, 2023

2023
[11]

Crayon: Customized on-device llm via instant adapter blending and edge-server hybrid inference

Jihwan Bang, Juntae Lee, Kyuhong Shim, et al. Crayon: Customized on-device llm via instant adapter blending and edge-server hybrid inference. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 3720–3731, 2024

2024
[13]

Ce-collm: Efficient and adaptive large language models through cloud-edge collaboration

Hongpeng Jin and Yanzhao Wu. Ce-collm: Efficient and adaptive large language models through cloud-edge collaboration. arXiv preprint arXiv:2411.02829, 2024

2024 arXiv
[14]

Hybrid slm and llm for edge-cloud collaborative inference

Zixu Hao, Huiqiang Jiang, Shiqi Jiang, et al. Hybrid slm and llm for edge-cloud collaborative inference. In Proc. Workshop Edge Mobile Found. Models, EdgeFM, pages 36–41, 2024

2024
[15]

Efficient deployment of large language model across cloud-device systems

Fan Yang, Zehao Wang, Haoyu Zhang, et al. Efficient deployment of large language model across cloud-device systems. In IEEE Int. Syst.-on-Chip Conf., SOCC, pages 1–6, 2024

2024
[16]

Gkt: A novel guidance-based knowledge transfer framework for efficient cloud-edge collaboration llm deployment

Yao Yao, Zuchao Li, and Hai Zhao. Gkt: A novel guidance-based knowledge transfer framework for efficient cloud-edge collaboration llm deployment. arXiv preprint arXiv:2405.19635, 2024

2024 arXiv
[17]

Large language models empowered autonomous edge ai for connected intelligence

Yifei Shen, Jiawei Shao, Xinjie Zhang, et al. Large language models empowered autonomous edge ai for connected intelligence. IEEE Commun. Mag., 62(10):140–146, 2024

2024
[18]

Moe2: Optimizing collaborative inference for edge large language models

Lyudong Jin, Yanning Zhang, Yanhan Li, et al. Moe2: Optimizing collaborative inference for edge large language models. arXiv preprint arXiv:2501.09410, 2025

2025 arXiv
[19]

Octo-planner: On-device language model for planner-action agents

Wei Chen, Zhiyuan Li, Zhen Guo, et al. Octo-planner: On-device language model for planner-action agents. arXiv preprint arXiv:2406.18082, 2024

2024 arXiv
[20]

A survey on model compression for large language models

Xunyu Zhu, Jian Li, Yong Liu, et al. A survey on model compression for large language models. Trans. Assoc. Comput. Linguistics , 12:1556–1577, 2024. , Vol. 1, No. 1, Article . Publication date: September 2025. Collaborative Inference and Learning between Edge SLMs and Cloud L...

2024
[21]

Pushing up to the limit of memory bandwidth and capacity utilization for efficient llm decoding on embedded fpga

Jindong Li, Tenglong Li, Guobin Shen, et al. Pushing up to the limit of memory bandwidth and capacity utilization for efficient llm decoding on embedded fpga. In Des. Autom. Test Eur. Conf., DATE, pages 1–7. IEEE, 2025

2025
[22]

Scaling up on-device llms via active-weight swapping between dram and flash.arXiv preprint arXiv:2504.08378, 2025

Fucheng Jia, Zewen Wu, Shiqi Jiang, et al. Scaling up on-device llms via active-weight swapping between dram and flash.arXiv preprint arXiv:2504.08378, 2025

2025
[23]

Understanding llms: A comprehensive overview from training to inference

Yiheng Liu, Hao He, Tianle Han, et al. Understanding llms: A comprehensive overview from training to inference. Neurocomputing, 620:129190, 2025

2025
[24]

Large models for aerial edges: An edge-cloud model evolution and communication paradigm

Shuhang Zhang, Qingyu Liu, Ke Chen, et al. Large models for aerial edges: An edge-cloud model evolution and communication paradigm. IEEE J. Sel. Areas Commun. , 43(1):21–35, 2025

2025
[26]

Llmcad: Fast and scalable on-device large language model inference

Daliang Xu, Wangsong Yin, Xin Jin, et al. Llmcad: Fast and scalable on-device large language model inference. arXiv preprint arXiv:2309.04255, 2023

2023 arXiv
[27]

Fuzzy speculative decoding for a tunable accuracy-runtime tradeoff

Maximilian Holsman, Yukun Huang, and Bhuwan Dhingra. Fuzzy speculative decoding for a tunable accuracy-runtime tradeoff. arXiv preprint arXiv:2502.20704, 2025

2025 arXiv
[28]

Duodecoding: Hardware-aware heterogeneous speculative decoding with dynamic multi-sequence drafting

Kai Lv, Honglin Guo, Qipeng Guo, and Xipeng Qiu. Duodecoding: Hardware-aware heterogeneous speculative decoding with dynamic multi-sequence drafting. arXiv preprint arXiv:2503.00784, 2025

2025 arXiv
[29]

Longspec: Long-context speculative decoding with efficient drafting and verification

Penghui Yang, Cunxiao Du, Fengzhuo Zhang, et al. Longspec: Long-context speculative decoding with efficient drafting and verification. arXiv preprint arXiv:2502.17421, 2025

2025 arXiv
[30]

Collaboration of large language models and small recommendation models for device-cloud recommendation

Zheqi Lv, Tianyu Zhan, Wenjie Wang, et al. Collaboration of large language models and small recommendation models for device-cloud recommendation. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , pages 962–973, 2025

2025
[31]

Crayon: Customized on-device LLM via instant adapter blending and edge-server hybrid inference

Jihwan Bang, Juntae Lee, Kyuhong Shim, et al. Crayon: Customized on-device LLM via instant adapter blending and edge-server hybrid inference. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 3720–3731, 2024

2024
[32]

Speculative decoding with big little decoder

Sehoon Kim, Karttikeya Mangalam, Suhong Moon, et al. Speculative decoding with big little decoder. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS, 2023

2023
[33]

Cogenesis: A framework collaborating large and small language models for secure context-aware instruction following

Kaiyan Zhang, Jianyu Wang, Remo Hua, et al. Cogenesis: A framework collaborating large and small language models for secure context-aware instruction following. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 4295–4312, 2024

2024
[35]

Cloud-edge collaborative large model services: Challenges and solutions

Yanghe Pan, Zhou Su, Yuntao Wang, et al. Cloud-edge collaborative large model services: Challenges and solutions. IEEE Netw., 2024

2024
[36]

Backpropagation-free multi-modal on-device model adaptation via cloud-device collaboration

Wei Ji, Li Li, Zheqi Lv, et al. Backpropagation-free multi-modal on-device model adaptation via cloud-device collaboration. ACM Trans. Multim. Comput. Commun. Appl., 21(2):69:1–69:17, 2025

2025
[37]

Reward-guided speculative decoding for efficient llm reasoning

Baohao Liao, Yuhui Xu, Hanze Dong, et al. Reward-guided speculative decoding for efficient llm reasoning. arXiv preprint arXiv:2501.19324, 2025

2025 arXiv
[38]

Kangaroo: Lossless self-speculative decoding for accelerating llms via double early exiting

Fangcheng Liu, Yehui Tang, Zhenhua Liu, et al. Kangaroo: Lossless self-speculative decoding for accelerating llms via double early exiting. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024
[39]

Adaserve: Slo-customized llm serving with fine-grained speculative decoding

Zikun Li, Zhuofu Chen, Remi Delacourt, et al. Adaserve: Slo-customized llm serving with fine-grained speculative decoding. arXiv preprint arXiv:2501.12162, 2025

2025 arXiv
[41]

Disco: Device-server collaborative llm-based text streaming services

Ting Sun, Penghan Wang, and Fan Lai. Disco: Device-server collaborative llm-based text streaming services. arXiv preprint arXiv:2502.11417, 2025

2025 arXiv
[43]

Pice: A semantic-driven progressive inference system for llm serving in cloud-edge networks

Huiyou Zhan, Xuan Zhang, Haisheng Tan, et al. Pice: A semantic-driven progressive inference system for llm serving in cloud-edge networks. arXiv preprint arXiv:2501.09367, 2025

2025 arXiv
[44]

Enhancing on-device llm inference with historical cloud-based llm interactions

Yucheng Ding, Chaoyue Niu, Fan Wu, et al. Enhancing on-device llm inference with historical cloud-based llm interactions. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , pages 597–608, 2024

2024
[45]

Velo: A vector database-assisted cloud-edge collaborative llm qos optimization framework

Zhi Yao, Zhiqing Tang, Jiong Lou, et al. Velo: A vector database-assisted cloud-edge collaborative llm qos optimization framework. In IEEE Int. Conf. Web Serv., ICWS , pages 865–876, 2024

2024
[46]

Fedmkt: Federated mutual knowledge transfer for large and small language models

Tao Fan, Guoqiang Ma, Yan Kang, et al. Fedmkt: Federated mutual knowledge transfer for large and small language models. In Proc. Int. Conf. Comput. Linguist., COLING , pages 243–255
[47]

Federated transfer learning for on-device llms efficient fine tuning optimization

Chuantao Li, Bruce Gu, Zhigang Zhao, et al. Federated transfer learning for on-device llms efficient fine tuning optimization. Big Data Min. Anal., 8(2):430–446, 2025

2025
[48]

Cloud-device collaborative learning for multimodal large language models

Guanqun Wang, Jiaming Liu, Chenxuan Li, et al. Cloud-device collaborative learning for multimodal large language models. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit., CVPR , pages 12646–12655, 2024. , Vol. 1, No. 1, Article . Publication date: September 2025. 28 • Senyao L...

2024
[49]

Division-of-thoughts: Harnessing hybrid language model synergy for efficient on-device agents

Chenyang Shao, Xinyuan Hu, Yutang Lin, et al. Division-of-thoughts: Harnessing hybrid language model synergy for efficient on-device agents. In Proc. ACM Web Conf., WWW , pages 1822–1833, 2025

2025
[50]

Edgemoe: Fast on-device inference of moe-based large language models

Rongjie Yi, Liwei Guo, Shiyun Wei, et al. Edgemoe: Fast on-device inference of moe-based large language models. arXiv preprint arXiv:2308.14352, 2023

2023 arXiv
[51]

Resource allocation for stable llm training in mobile edge computing

Chang Liu and Jun Zhao. Resource allocation for stable llm training in mobile edge computing. In Proc. ACM Int. Symp. Mobile Ad Hoc Netw. Comput., MOBIHOC, pages 81–90, 2024

2024
[52]

Richard Yu, et al

Ying He, Jingcheng Fang, F. Richard Yu, et al. Large language models (llms) inference offloading and resource allocation in cloud-edge computing: An active inference approach. IEEE Trans. Mob. Comput., 23(12):11253–11264, 2024

2024
[53]

Adaptlink: A heterogeneity-aware adaptive framework for distributed mllm inference

Xinyi Hu, Zihan Chen, Kun Guo, et al. Adaptlink: A heterogeneity-aware adaptive framework for distributed mllm inference. In Proc. AAAI 2025 Workshop Artif. Intell. Wireless Commun. Netw., AI4WCN , 2025

2025
[54]

Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction

Bhaskarjit Sarmah, Dhagash Mehta, Benika Hall, et al. Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. In Proc. 5th ACM Int. Conf. AI Finance, ICAIF , pages 608–616, 2024

2024
[56]

Edge-cloud collaborative motion planning for autonomous driving with large language models

Jiao Chen, Suyan Dai, Fangfang Chen, et al. Edge-cloud collaborative motion planning for autonomous driving with large language models. In Proc. IEEE ICCT, ICCT, pages 185–190, 2024

2024
[57]

Grey-box prompt optimization and fine-tuning for cloud-edge LLM agents, 2024

Ya Liu, Kai Yang, Yu Zhu, et al. Grey-box prompt optimization and fine-tuning for cloud-edge LLM agents, 2024

2024
[58]

Small models are valuable plug-ins for large language models

Canwen Xu, Yichong Xu, Shuohang Wang, et al. Small models are valuable plug-ins for large language models. arXiv preprint arXiv:2305.08848, 2023

2023 arXiv
[59]

Slim: Speculative decoding with hypothesis reduction

Chi-Heng Lin, Shikhar Tuli, James Seale Smith, et al. Slim: Speculative decoding with hypothesis reduction. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , pages 1005–1017, 2024

2024
[60]

Opt-tree: Speculative decoding with adaptive draft tree structure

Jikai Wang, Yi Su, Juntao Li, et al. Opt-tree: Speculative decoding with adaptive draft tree structure. Trans. Assoc. Comput. Linguistics , 13:188–199, 2025

2025
[62]

Speculative decoding via early-exiting for faster llm inference with thompson sampling control mechanism

Jiahao Liu, Qifan Wang, Jingang Wang, et al. Speculative decoding via early-exiting for faster llm inference with thompson sampling control mechanism. In Findings Assoc. Comput. Linguist., ACL , pages 3027–3043
[63]

Diffusion-based cloud-edge-device collaborative learning for next poi recommendations

Jing Long, Guanhua Ye, Tong Chen, et al. Diffusion-based cloud-edge-device collaborative learning for next poi recommendations. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , pages 2026–2036, 2024

2026
[64]

An edge-cloud collaboration framework for generative AI service provision with synergetic big cloud model and small edge models

Yuqing Tian, Zhaoyang Zhang, Yuzhi Yang, et al. An edge-cloud collaboration framework for generative AI service provision with synergetic big cloud model and small edge models. IEEE Netw., 38(5):37–46, 2024

2024
[65]

Enhanced hybrid inference techniques for scalable on-device llm personalization and cloud integration

Teresa Peng, Liam Liu, Maya Gupta, et al. Enhanced hybrid inference techniques for scalable on-device llm personalization and cloud integration. contexts, 8(17):18–25, 2024

2024
[66]

Edge-llm: A collaborative framework for large language model serving in edge computing

Fenglong Cai, Dong Yuan, Zhe Yang, et al. Edge-llm: A collaborative framework for large language model serving in edge computing. In IEEE Int. Conf. Web Serv., ICWS , pages 799–809, 2024

2024
[67]

Mergenet: Knowledge migration across heterogeneous models, tasks, and modalities

Kunxi Li, Tianyu Zhan, Kairui Fu, et al. Mergenet: Knowledge migration across heterogeneous models, tasks, and modalities. In Proc. AAAI Conf. Artif. Intell., AAAI , volume 39, pages 4824–4832, 2025

2025
[68]

Optimize incompatible parameters through compatibility-aware knowledge integration

Zheqi Lv, Keming Ye, Zishu Wei, et al. Optimize incompatible parameters through compatibility-aware knowledge integration. In Proc. AAAI Conf. Artif. Intell., AAAI , pages 19233–19241, 2025

2025
[69]

Forward once for all: Structural parameterized adaptation for efficient cloud-coordinated on-device recommendation

Kairui Fu, Zheqi Lv, Shengyu Zhang, et al. Forward once for all: Structural parameterized adaptation for efficient cloud-coordinated on-device recommendation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD , pages 318–329, 2025

2025
[70]

Diet: Customized slimming for incompatible networks in sequential recommendation

Kairui Fu, Shengyu Zhang, Zheqi Lv, et al. Diet: Customized slimming for incompatible networks in sequential recommendation. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , pages 816–826, 2024

2024
[71]

Edge vs cloud: How do we balance cost, latency, and quality for large language models over 5g networks? In Proc

Minsu Kim, Pinyarash Pinyoanuntapong, Bong-Ho Kim, et al. Edge vs cloud: How do we balance cost, latency, and quality for large language models over 5g networks? In Proc. IEEE Wireless Commun. Netw. Conf., WCNC , pages 1–6, 2025

2025
[72]

Edge-cloud collaborative computing on distributed intelligence and model optimization: A survey

Jing Liu, Yao Du, Kun Yang, et al. Edge-cloud collaborative computing on distributed intelligence and model optimization: A survey. arXiv preprint arXiv:2505.01821, 2025

2025
[73]

Fedcfa: Alleviating simpson’s paradox in model aggregation with counterfactual federated learning

Zhonghua Jiang, Jimin Xu, Shengyu Zhang, et al. Fedcfa: Alleviating simpson’s paradox in model aggregation with counterfactual federated learning. In Proc. AAAI Conf. Artif. Intell., AAAI , pages 17662–17670, 2025

2025
[74]

Modelgpt: Unleashing llm’s capabilities for tailored model generation

Zihao Tang, Zheqi Lv, Shengyu Zhang, et al. Modelgpt: Unleashing llm’s capabilities for tailored model generation. arXiv preprint arXiv:2402.12408, 2024

2024 arXiv
[75]

Intelligent model update strategy for sequential recommendation

Zheqi Lv, Wenqiao Zhang, Zhengyu Chen, et al. Intelligent model update strategy for sequential recommendation. In Proc. ACM Web Conf., WWW, pages 3117–3128, 2024. , Vol. 1, No. 1, Article . Publication date: September 2025. Collaborative Inference and Learning between Edge SLM...

2024
[76]

Aug-kd: Anchor-based mixup generation for out-of-domain knowledge distillation

Zihao Tang, Zheqi Lv, Shengyu Zhang, et al. Aug-kd: Anchor-based mixup generation for out-of-domain knowledge distillation. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[77]

Llmco4mr: Llms-aided neural combinatorial optimization for ancient manuscript restoration from fragments with case studies on dunhuang

Yuqing Zhang, Hangqi Li, Shengyu Zhang, et al. Llmco4mr: Llms-aided neural combinatorial optimization for ancient manuscript restoration from fragments with case studies on dunhuang. In Proc. Eur. Conf. Comput. Vis., ECCV , volume 15133, pages 253–269, 2024

2024
[78]

Mpod123: One image to 3d content generation using mask-enhanced progressive outline-to-detail optimization

Jimin Xu, Tianbao Wang, Tao Jin, et al. Mpod123: One image to 3d content generation using mask-enhanced progressive outline-to-detail optimization. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit., CVPR , pages 10682–10692, 2024

2024
[79]

Fedcollm: A parameter-efficient federated co-tuning framework for large and small language models

Tao Fan, Yan Kang, Guoqiang Ma, et al. Fedcollm: A parameter-efficient federated co-tuning framework for large and small language models. arXiv preprint arXiv:2411.11707, 2024

2024 arXiv
[80]

Llm-empowered embodied agent for memory-augmented task planning in household robotics

Marc Glocker, Peter Hönig, Matthias Hirschmanner, et al. Llm-empowered embodied agent for memory-augmented task planning in household robotics. arXiv preprint arXiv:2504.21716, 2025

2025 arXiv
[82]

Mehdi Mojarradi, Lingyi Yang, Robert McCraith, et al

M. Mehdi Mojarradi, Lingyi Yang, Robert McCraith, et al. Improving in-context learning with small language model ensembles. arXiv preprint arXiv:2410.21868, 2024

2024 arXiv
[83]

Collab-rag: Boosting retrieval-augmented generation for complex question answering via white-box and black-box llm collaboration

Ran Xu, Wenqi Shi, Yuchen Zhuang, et al. Collab-rag: Boosting retrieval-augmented generation for complex question answering via white-box and black-box llm collaboration. arXiv preprint arXiv:2504.04915, 2025

2025 arXiv
[85]

Knowledge-decoupled synergetic learning: An MLLM based collaborative approach to few-shot multimodal dialogue intention recognition

Bin Chen, Yu Zhang, Hongfei Ye, et al. Knowledge-decoupled synergetic learning: An MLLM based collaborative approach to few-shot multimodal dialogue intention recognition. In Proc. ACM Web Conf., WWW , pages 3044–3048, 2025

2025
[86]

Confident or seek stronger: Exploring uncertainty-based on-device llm routing from benchmarking to generalization

Yu-Neng Chuang, Leisheng Yu, Guanchu Wang, et al. Confident or seek stronger: Exploring uncertainty-based on-device llm routing from benchmarking to generalization. arXiv preprint arXiv:2502.04428, 2025

2025 arXiv
[87]

Mutual enhancement of large and small language models with cross-silo knowledge transfer

Yongheng Deng, Ziqing Qiao, Ju Ren, et al. Mutual enhancement of large and small language models with cross-silo knowledge transfer. arXiv preprint arXiv:2312.05842, 2023

2023 arXiv
[88]

Modular pluralism: Pluralistic alignment via multi-llm collaboration

Shangbin Feng, Taylor Sorensen, Yuhan Liu, et al. Modular pluralism: Pluralistic alignment via multi-llm collaboration. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 4151–4171, 2024

2024
[89]

Efficient multitask learning in small language models through upside-down reinforcement learning

Yu-Chen Lin, Sanat Sharma, Hari Manikandan, et al. Efficient multitask learning in small language models through upside-down reinforcement learning. arXiv preprint arXiv:2502.09854, 2025

2025
[90]

Towards harnessing the collaborative power of large and small models for domain tasks

Yang Liu, Bingjie Yan, Tianyuan Zou, et al. Towards harnessing the collaborative power of large and small models for domain tasks. arXiv preprint arXiv:2504.17421, 2025

2025 arXiv
[91]

Slmrec: Distilling large language models into small for sequential recommendation

Wujiang Xu, Qitian Wu, Zujie Liang, et al. Slmrec: Distilling large language models into small for sequential recommendation. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[92]

Automix: Automatically mixing language models

Pranjal Aggarwal, Aman Madaan, Ankit Anand, et al. Automix: Automatically mixing language models. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS, 2024

2024
[93]

Weak-to-strong generalization: Eliciting strong capabilities with weak supervision

Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, et al. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. In Proc. Int. Conf. Mach. Learn., ICML , 2024

2024
[94]

Routerdc: Query-based router by dual contrastive learning for assembling large language models

Shuhao Chen, Weisen Jiang, Baijiong Lin, et al. Routerdc: Query-based router by dual contrastive learning for assembling large language models. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024
[95]

Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors

Weize Chen, Yusheng Su, Jingwei Zuo, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[96]

Heterogeneous lora for federated fine-tuning of on-device foundation models

Yae Jee Cho, Luyang Liu, Zheng Xu, et al. Heterogeneous lora for federated fine-tuning of on-device foundation models. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 12903–12913, 2024

2024
[97]

Hybrid llm: Cost-efficient and quality-aware query routing

Dujian Ding, Ankur Mallick, Chi Wang, et al. Hybrid llm: Cost-efficient and quality-aware query routing. In Proc. 12th Int. Conf. Learn. Represent., ICLR, 2024

2024
[98]

Data shunt: Collaboration of small and large models for lower costs and better performance

Dong Chen, Yueting Zhuang, Shuo Zhang, et al. Data shunt: Collaboration of small and large models for lower costs and better performance. In Proc. AAAI Conf. Artif. Intell., AAAI , volume 38, pages 11249–11257, 2024

2024
[99]

Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes

Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, et al. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings Assoc. Comput. Linguist., ACL , pages 8003–8017, 2023

2023
[100]

Fast inference from transformers via speculative decoding

Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Proc. Int. Conf. Mach. Learn., ICML, 2023

2023
[101]

Ddk: Distilling domain knowledge for efficient large language models

Jiaheng Liu, Chenchen Zhang, Jinyang Guo, et al. Ddk: Distilling domain knowledge for efficient large language models. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024
[102]

Co-supervised learning: Improving weak-to-strong generalization with hierarchical mixture of experts

Yuejiang Liu and Alexandre Alahi. Co-supervised learning: Improving weak-to-strong generalization with hierarchical mixture of experts. arXiv preprint arXiv:2402.15505, 2024. , Vol. 1, No. 1, Article . Publication date: September 2025. 30 • Senyao Li, Haozhao Wang, Wenchao Xu,...

2024 arXiv
[103]

Llm-qat: Data-free quantization aware training for large language models

Zechun Liu, Barlas Oguz, Changsheng Zhao, et al. Llm-qat: Data-free quantization aware training for large language models. In Findings Assoc. Comput. Linguist., ACL , pages 467–484, 2024

2024
[104]

Duet: A tuning-free device-cloud collaborative parameters generation framework for efficient device model generalization

Zheqi Lv, Wenqiao Zhang, Shengyu Zhang, et al. Duet: A tuning-free device-cloud collaborative parameters generation framework for efficient device model generalization. In Proc. ACM Web Conf., WWW , pages 3077–3085, 2023

2023
[105]

An emulator for fine-tuning large language models using small language models

Eric Mitchell, Rafael Rafailov, Archit Sharma, et al. An emulator for fine-tuning large language models using small language models. In Proc. 12th Int. Conf. Learn. Represent., ICLR , page 16, 2024

2024
[106]

Routellm: Learning to route llms from preference data

Isaac Ong, Amjad Almahairi, Vincent Wu, et al. Routellm: Learning to route llms from preference data. In Proc. 13th Int. Conf. Learn. Represent., ICLR, page 16, 2025

2025
[107]

A little help goes a long way: Efficient llm training by leveraging small lms

Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, et al. A little help goes a long way: Efficient llm training by leveraging small lms. arXiv preprint arXiv:2410.18779, 2024

2024 arXiv
[108]

Your student is better than expected: Adaptive teacher-student collaboration for text-conditional diffusion models

Nikita Starodubcev, Dmitry Baranchuk, Artem Fedorov, et al. Your student is better than expected: Adaptive teacher-student collaboration for text-conditional diffusion models. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit., CVPR , pages 9275–9285, 2024

2024
[109]

Speculative decoding: Exploiting speculative execution for accelerating seq2seq generation

Heming Xia, Tao Ge, Peiyi Wang, et al. Speculative decoding: Exploiting speculative execution for accelerating seq2seq generation. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 3909–3925, 2023

2023
[110]

Sheared llama: Accelerating language model pre-training via structured pruning

Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, et al. Sheared llama: Accelerating language model pre-training via structured pruning. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[112]

Revisiting knowledge distillation for autoregressive language models

Qihuang Zhong, Liang Ding, Li Shen, et al. Revisiting knowledge distillation for autoregressive language models. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 10900–10913, 2024

2024
[113]

Llm for mobile: An initial roadmap

Daihang Chen, Yonghui Liu, Mingyi Zhou, et al. Llm for mobile: An initial roadmap. ACM Trans. Softw. Eng. Methodol., 34(5), 2025

2025
[114]

Hybrid-raca: Hybrid retrieval-augmented composition assistance for real-time text prediction

Menglin Xia, Xuchao Zhang, Camille Couturier, et al. Hybrid-raca: Hybrid retrieval-augmented composition assistance for real-time text prediction. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 120–131, 2024

2024
[115]

Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services

Zheming Yang, Yuanhao Yang, Chang Zhao, et al. Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services. arXiv preprint arXiv:2405.14636, 2024

2024 arXiv
[118]

Beyond the cloud: Edge inference for generative large language models in wireless networks

Xinyuan Zhang, Jiangtian Nie, Yudong Huang, et al. Beyond the cloud: Edge inference for generative large language models in wireless networks. IEEE Trans. Wireless Commun., 24(1):643–658, 2025

2025
[119]

A cloud-edge collaborative architecture for multimodal llm-based advanced driver assistance systems in iot networks

Yaqi Hu, Dongdong Ye, Jiawen Kang, et al. A cloud-edge collaborative architecture for multimodal llm-based advanced driver assistance systems in iot networks. IEEE Internet Things J. , 12(10):13208–13221, 2025

2025
[120]

Cloud-edge collaborative inference with network pruning

Mingran Li, Xuejun Zhang, Jiasheng Guo, et al. Cloud-edge collaborative inference with network pruning. Electronics, 12(17):3598, 2023

2023
[121]

Efficientllm: Scalable pruning-aware pretraining for architecture-agnostic edge language models

Xingrun Xing, Zheng Liu, Shitao Xiao, et al. Efficientllm: Scalable pruning-aware pretraining for architecture-agnostic edge language models. arXiv preprint arXiv:2502.06663, 2025

2025 arXiv
[122]

High-efficiency device-cloud collaborative transformer model

Penghao Jiang, Ke Xin, Chunxi Li, et al. High-efficiency device-cloud collaborative transformer model. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit., CVPR, pages 2204–2210, 2023

2023
[123]

Context-aware layer scheduling for seamless neural network inference in cloud-edge systems

Matthias Stammler, Vladimir Sidorenko, Fabian Kreß, et al. Context-aware layer scheduling for seamless neural network inference in cloud-edge systems. In IEEE 16th Int. Symp. Embedded Multicore/Many-core Syst.-on-Chip,MCSoC , pages 97–104, 2023

2023
[125]

Auto-tuning neural network quantization framework for collaborative inference between the cloud and edge

Guangli Li, Lei Liu, Xueying Wang, et al. Auto-tuning neural network quantization framework for collaborative inference between the cloud and edge. In Int. Conf. Artif. Neural Netw. Mach. Learn., ICANN , volume 11139, pages 402–411, 2018

2018
[126]

Ouroboros: Generating longer drafts phrase by phrase for faster speculative decoding

Weilin Zhao, Yuxiang Huang, Xu Han, et al. Ouroboros: Generating longer drafts phrase by phrase for faster speculative decoding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 13378–13393, 2024

2024
[128]

Accelerating llm inference with staged speculative decoding

Benjamin Spector and Chris Re. Accelerating llm inference with staged speculative decoding. arXiv preprint arXiv:2308.04623, 2023

2023 arXiv
[129]

Towards integrated fine-tuning and inference when generative ai meets edge intelligence

Ning Chen, Zhipeng Cheng, Xuwei Fan, et al. Towards integrated fine-tuning and inference when generative ai meets edge intelligence. arXiv preprint arXiv:2401.02668, 2024

2024 arXiv
[130]

Layerskip: Enabling early exit inference and self-speculative decoding

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, et al. Layerskip: Enabling early exit inference and self-speculative decoding. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 12622–12642, 2024. , Vol. 1, No. 1, Article . Publication date: S...

2024
[131]

Specinfer: Accelerating large language model serving with tree-based speculative inference and verification

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al. Specinfer: Accelerating large language model serving with tree-based speculative inference and verification. In Proc. 29th ACM Int. Conf. Archit. Support Program. Lang. Oper. Syst., vol. 3, ASPLOS , pages 932–949, 2024

2024
[132]

Federated reinforcement learning-empowered task offloading for large models in vehicular edge computing

Huaming Wu, Anqi Gu, and Yonghui Liang. Federated reinforcement learning-empowered task offloading for large models in vehicular edge computing. IEEE Trans. Veh. Technol., 74(2):1979–1991, 2025

1979
[133]

Traversal verification for speculative tree decoding

Yepeng Weng, Qiao Hu, Xujie Chen, et al. Traversal verification for speculative tree decoding. arXiv preprint arXiv:2505.12398, 2025

2025
[134]

Sequoia: Scalable and robust speculative decoding

Zhuoming Chen, Avner May, Ruslan Svirschevski, et al. Sequoia: Scalable and robust speculative decoding. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024
[135]

Pipespec: Breaking stage dependencies in hierarchical llm decoding

Bradley McDanel, Sai Qian Zhang, Yunhai Hu, et al. Pipespec: Breaking stage dependencies in hierarchical llm decoding. arXiv preprint arXiv:2505.01572, 2025

2025 arXiv
[136]

Hamburger: Accelerating llm inference via token smashing

Jingyu Liu and Ce Zhang. Hamburger: Accelerating llm inference via token smashing. arXiv preprint arXiv:2505.20438, 2025

2025 arXiv
[137]

Distillspec: Improving speculative decoding via knowledge distillation

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, et al. Distillspec: Improving speculative decoding via knowledge distillation. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[138]

Minions: Cost-efficient collaboration between on-device and cloud language models

Avanika Narayan, Dan Biderman, Sabri Eyuboglu, et al. Minions: Cost-efficient collaboration between on-device and cloud language models. arXiv preprint arXiv:2502.15964, 2025

2025 arXiv
[139]

Gemini: A family of highly capable multimodal models

Gemini Team, Rohan Anil, Sebastian Borgeaud, et al. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2025

2025 arXiv
[140]

Phi-3 technical report: A highly capable language model locally on your phone

Marah Abdin, Jyoti Aneja, Hany Awadalla, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

2024 arXiv
[141]

Gpt-4o system card

OpenAI, Aaron Hurst, Adam Lerer, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

2024 arXiv
[142]

Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet

S Anthropic. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. URL https://api. semanticscholar. org/CorpusID, 273639283
[143]

General data protection regulation

Protection Regulation. General data protection regulation. Intouch, 25:1–5, 2018

2018
[144]

Palm: Scaling language modeling with pathways

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al. Palm: Scaling language modeling with pathways. J. Mach. Learn. Res. , 24:240:1–240:113, 2023

2023
[145]

Leaf: A benchmark for federated settings

Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, et al. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2019

2019 arXiv
[146]

Federated visual classification with real-world data distribution

Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Federated visual classification with real-world data distribution. In Proc. Eur. Conf. Comput. Vis., ECCV , volume 12355, pages 76–92, 2020

2020
[147]

Personalized dialogue generation with diversified traits.arXiv preprint arXiv:1901.09672, 2019

Yinhe Zheng, Guanyi Chen, Minlie Huang, et al. Personalized dialogue generation with diversified traits.arXiv preprint arXiv:1901.09672, 2019

1901 arXiv
[148]

Personalizing dialogue agents: I have a dog, do you have pets too? In Proc

Saizheng Zhang, Emily Dinan, Jack Urbanek, et al. Personalizing dialogue agents: I have a dog, do you have pets too? In Proc. 56th Annu. Meet. Assoc. Comput. Linguist., ACL , pages 2204–2213, 2018

2018
[149]

Livechat: A large-scale personalized dialogue dataset automatically constructed from live streaming

Jingsheng Gao, Yixin Lian, Ziyi Zhou, et al. Livechat: A large-scale personalized dialogue dataset automatically constructed from live streaming. In Proc. 61st Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 15387–15405, 2023

2023
[150]

Fedscale: Benchmarking model and system performance of federated learning at scale

Fan Lai, Yinwei Dai, Sanjay Sri Vallabh Singapuram, et al. Fedscale: Benchmarking model and system performance of federated learning at scale. In Proc. Int. Conf. Mach. Learn., ICML , volume 162, pages 11814–11827, 2022

2022
[151]

FedNLP: Benchmarking federated learning methods for natural language processing tasks

Bill Yuchen Lin, Chaoyang He, Zihang Ze, et al. FedNLP: Benchmarking federated learning methods for natural language processing tasks. In Findings Assoc. Comput. Linguist., NAACL , pages 157–175, 2022

2022
[152]

Fedmultimodal: A benchmark for multimodal federated learning

Tiantian Feng, Digbalay Bose, Tuo Zhang, et al. Fedmultimodal: A benchmark for multimodal federated learning. InProc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , page 4035–4045, 2023

2023
[153]

pfl-bench: A comprehensive benchmark for personalized federated learning

Daoyuan Chen, Dawei Gao, Weirui Kuang, et al. pfl-bench: A comprehensive benchmark for personalized federated learning. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2022

2022
[154]

On-device model fine-tuning with label correction in recommender systems

Yucheng Ding, Chaoyue Niu, Fan Wu, et al. On-device model fine-tuning with label correction in recommender systems. arXiv preprint arXiv:2211.01163, 2022

2022 arXiv
[155]

Optimized cost per click in taobao display advertising

Han Zhu, Junqi Jin, Chang Tan, et al. Optimized cost per click in taobao display advertising. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD, page 2191–2200, 2017

2017
[156]

Ml-enabled systems model deployment and monitoring: Status quo and problems

Eduardo Zimelewicz, Marcos Kalinowski, Daniel Méndez, et al. Ml-enabled systems model deployment and monitoring: Status quo and problems. In 16th Int. Conf. Softw. Qual., SWQD , volume 505, pages 112–131, 2024

2024
[158]

Bonawitz, Hubert Eichner, Wolfgang Grieskamp, et al

Kallista A. Bonawitz, Hubert Eichner, Wolfgang Grieskamp, et al. Towards federated learning at scale: System design. In Proc. 2nd Conf. Mach. Learn. Syst., SysML , 2019

2019
[159]

Fedml: A research library and benchmark for federated machine learning

Chaoyang He, Songze Li, Jinhyun So, et al. Fedml: A research library and benchmark for federated machine learning. 2020

2020
[160]

Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning

Zhen Wang, Weirui Kuang, Yuexiang Xie, et al. Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD , pages 4110–4120, 2022. , Vol. 1, No. 1, Article . Publication date: ...

2022
[161]

Beutel, Taner Topal, Akhil Mathur, et al

Daniel J. Beutel, Taner Topal, Akhil Mathur, et al. Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390, 2022

2007 arXiv
[164]

Transitioning from mlops to llmops: Navigating the unique challenges of large language models

Saurabh Pahune and Zahid Akhtar. Transitioning from mlops to llmops: Navigating the unique challenges of large language models. Information, 16(2), 2025

2025
[165]

Edge-cloud routing for text-to-image model with token-level multi-metric prediction

Zewei Xin, Qinya Li, Chaoyue Niu, et al. Edge-cloud routing for text-to-image model with token-level multi-metric prediction. arXiv preprint arXiv:2411.13787, 2024

2024 arXiv
[166]

Routing to the expert: Efficient reward-guided ensemble of large language models

Keming Lu, Hongyi Yuan, Runji Lin, et al. Routing to the expert: Efficient reward-guided ensemble of large language models. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , pages 1964–1974, 2024

2024
[167]

Frugalgpt: How to use large language models while reducing cost and improving performance

Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. Trans. Mach. Learn. Res. , 2024

2024
[168]

Tabi: An efficient multi-level inference system for large language models

Yiding Wang, Kai Chen, Haisheng Tan, et al. Tabi: An efficient multi-level inference system for large language models. In Proc. 18th Eur. Conf. Comput. Syst., EuroSys , pages 233–248, 2023

2023
[169]

MixLLM: Dynamic routing in mixed large language models

Xinyuan Wang, Yanchi Liu, Wei Cheng, et al. MixLLM: Dynamic routing in mixed large language models. In Proc. 2025 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , pages 10912–10922, 2025

2025
[170]

Self-knowledge guided retrieval augmentation for large language models

Yile Wang, Peng Li, Maosong Sun, et al. Self-knowledge guided retrieval augmentation for large language models. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 10303–10315, 2023

2023
[171]

Self-RAG: Learning to retrieve, generate, and critique through self-reflection

Akari Asai, Zeqiu Wu, Yizhong Wang, et al. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[172]

Corrective retrieval augmented generation

Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, et al. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884, 2024

2024 arXiv
[173]

Ra-isf: Learning to answer and understand from retrieval augmentation via iterative self-feedback

Yanming Liu, Xinyue Peng, Xuhong Zhang, et al. Ra-isf: Learning to answer and understand from retrieval augmentation via iterative self-feedback. In Findings Assoc. Comput. Linguist., ACL , pages 4730–4749, 2024

2024
[174]

Small models, big insights: Leveraging slim proxy models to decide when and what to retrieve for llms

Jiejun Tan, Zhicheng Dou, Yutao Zhu, et al. Small models, big insights: Leveraging slim proxy models to decide when and what to retrieve for llms. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 4420–4436, 2024

2024
[175]

Toolformer: language models can teach themselves to use tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, et al. Toolformer: language models can teach themselves to use tools. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2023

2023
[176]

Knowledge card: Filling llms’ knowledge gaps with plug-in specialized language models

Shangbin Feng, Weijia Shi, Yuyang Bai, et al. Knowledge card: Filling llms’ knowledge gaps with plug-in specialized language models. In Proc. 11th Int. Conf. Learn. Represent., ICLR , 2023

2023
[177]

Purifying large language models by ensembling a small language model

Tianlin Li, Qian Liu, Tianyu Pang, et al. Purifying large language models by ensembling a small language model. arXiv preprint arXiv:2402.14845, 2024

2024 arXiv
[178]

Smaller language models are capable of selecting instruction-tuning training data for larger language models

Dheeraj Mekala, Alex Nguyen, and Jingbo Shang. Smaller language models are capable of selecting instruction-tuning training data for larger language models. In Findings Assoc. Comput. Linguist., ACL , pages 10456–10470, 2024

2024
[179]

Small language models as effective guides for large language models in chinese relation extraction

Xuemei Tang and Jun Wang. Small language models as effective guides for large language models in chinese relation extraction. arXiv preprint arXiv:2402.14373, 2024

2024 arXiv
[180]

Enhancing empathetic response generation by augmenting llms with small-scale empathetic models

Zhou Yang, Zhaochun Ren, Wang Yufeng, et al. Enhancing empathetic response generation by augmenting llms with small-scale empathetic models. arXiv preprint arXiv:2402.11801, 2024

2024 arXiv
[181]

Blade: Enhancing black-box large language models with small domain-specific models

Haitao Li, Qingyao Ai, Jia Chen, et al. Blade: Enhancing black-box large language models with small domain-specific models. In Proc. AAAI Conf. Artif. Intell., AAAI , volume 39, pages 24422–24430, 2025

2025
[182]

SLIDE: A framework integrating small and large language models for open-domain dialogues evaluation

Kun Zhao, Bohao Yang, Chen Tang, et al. SLIDE: A framework integrating small and large language models for open-domain dialogues evaluation. In Findings Assoc. Comput. Linguist., ACL , pages 15421–15435, 2024

2024
[183]

Citer: Collaborative inference for efficient large language model decoding with token-level routing

Wenhao Zheng, Yixiao Chen, Weitong Zhang, et al. Citer: Collaborative inference for efficient large language model decoding with token-level routing. arXiv preprint arXiv:2502.01976, 2025

2025 arXiv
[184]

Device-unimodal cloud-multimodal collaboration for livestreaming content understanding

Yufei Zhu, Chaoyue Niu, Yikai Yan, et al. Device-unimodal cloud-multimodal collaboration for livestreaming content understanding. In IEEE Int. Conf. Data Min., ICDM , pages 1571–1576, 2023

2023
[185]

Eaco-rag: Towards distributed tiered llm deployment using edge-assisted and collaborative rag with adaptive knowledge update

Jiaxing Li, Chi Xu, Lianchen Jia, et al. Eaco-rag: Towards distributed tiered llm deployment using edge-assisted and collaborative rag with adaptive knowledge update. arXiv preprint arXiv:2410.20299, 2025

2025 arXiv
[186]

T-rag: Lessons from the llm trenches

Masoomali Fatehkia, Ji Kim Lucas, and Sanjay Chawla. T-rag: Lessons from the llm trenches. arXiv preprint arXiv:2402.07483, 2024

2024 arXiv
[187]

Pushing large language models to the 6g edge: Vision, challenges, and opportunities, 2023

Zheng Lin, Guanqiao Qu, Qiyuan Chen, Xianhao Chen, Zhe Chen, and Kaibin Huang. Pushing large language models to the 6g edge: Vision, challenges, and opportunities, 2023

2023
[188]

Empirical guidelines for deploying llms onto resource-constrained edge devices

Ruiyang Qin, Dancheng Liu, Chenhui Xu, et al. Empirical guidelines for deploying llms onto resource-constrained edge devices. ACM Trans. Des. Autom. Electron. Syst. , 2025. , Vol. 1, No. 1, Article . Publication date: September 2025. Collaborative Inference and Learning betwee...

2025
[190]

Robust implementation of retrieval-augmented generation on edge-based computing-in- memory architectures

Ruiyang Qin, Zheyu Yan, Dewen Zeng, et al. Robust implementation of retrieval-augmented generation on edge-based computing-in- memory architectures. In Proc. 43rd IEEE/ACM Int. Conf. Comput.-Aided Des., ICCAD , 2025

2025
[191]

Pearl: Parallel speculative decoding with adaptive draft length

Tianyu Liu, Yun Li, Qitan Lv, et al. Pearl: Parallel speculative decoding with adaptive draft length. In Proc. 13th Int. Conf. Learn. Represent., ICLR, 2025

2025
[192]

Facil: Flexible DRAM address mapping for soc-pim cooperative on-device LLM inference

Seong Hoon Seo, Junghoon Kim, Donghyun Lee, et al. Facil: Flexible DRAM address mapping for soc-pim cooperative on-device LLM inference. In IEEE Int. Symp. High Perform. Comput. Archit., HPCA , pages 1720–1733, 2025

2025
[193]

Lincoln: Real-time 50~100b LLM inference on consumer devices with lpddr-interfaced, compute-enabled flash memory

Weiyi Sun, Mingyu Gao, Zhaoshi Li, et al. Lincoln: Real-time 50~100b LLM inference on consumer devices with lpddr-interfaced, compute-enabled flash memory. In IEEE Int. Symp. High Perform. Comput. Archit., HPCA , pages 1734–1750. IEEE, 2025

2025
[194]

Falcon: Faster and parallel inference of large language models through enhanced semi-autoregressive drafting and custom-designed decoding tree

Xiangxiang Gao, Weisheng Xie, Yiwei Xiang, et al. Falcon: Faster and parallel inference of large language models through enhanced semi-autoregressive drafting and custom-designed decoding tree. In Proc. AAAI Conf. Artif. Intell., AAAI , pages 23933–23941, 2025

2025
[195]

Hierarchical speculative decoding with dynamic window

Shensian Syu and Hung-yi Lee. Hierarchical speculative decoding with dynamic window. In Proc. 2025 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , 2025

2025
[196]

A bargaining game for personalized, energy efficient split learning over wireless networks

Minsu Kim, Alexander DeRieux, and Walid Saad. A bargaining game for personalized, energy efficient split learning over wireless networks. In Proc. IEEE Wireless Commun. Netw. Conf., WCNC , pages 1–6, 2023

2023
[197]

Retrieval augmented generation on hybrid cloud: A new architecture for knowledge base systems

Chia-Chuan Chuang and Kai-Ching Chen. Retrieval augmented generation on hybrid cloud: A new architecture for knowledge base systems. In 16th IIAI Int. Congr. Adv. Appl. Inform.,IIAI-AAI, pages 68–71, 2024

2024
[198]

A collaborative multi-agent approach to retrieval-augmented generation across diverse data

Aniruddha Salve, Saba Attar, Mahesh Deshmukh, et al. A collaborative multi-agent approach to retrieval-augmented generation across diverse data. arXiv preprint arXiv:2412.05838, 2024

2024 arXiv
[199]

CoMM: Collaborative multi-agent, multi-reasoning-path prompting for complex problem solving

Pei Chen, Shuai Zhang, and Boran Han. CoMM: Collaborative multi-agent, multi-reasoning-path prompting for complex problem solving. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL, pages 1720–1738, 2024

2024
[200]

Longrag: Enhancing retrieval-augmented generation with long-context llms

Ziyan Jiang, Xueguang Ma, and Wenhu Chen. Longrag: Enhancing retrieval-augmented generation with long-context llms. arXiv preprint arXiv:2406.15319, 2024

2024 arXiv
[201]

Nearest neighbor speculative decoding for llm generation and attribution

Minghan Li, Xilun Chen, Ari Holtzman, et al. Nearest neighbor speculative decoding for llm generation and attribution. Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 37:80987–81015, 2024

2024
[202]

Cloud-device collaborative adaptation to continual changing environments in the real-world

Yulu Gan, Mingjie Pan, Rongyu Zhang, et al. Cloud-device collaborative adaptation to continual changing environments in the real-world. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit., CVPR , pages 12157–12166, 2023

2023
[203]

Adaptive layer splitting for wireless llm inference in edge computing: A model-based reinforcement learning approach

Yuxuan Chen, Rongpeng Li, Xiaoxue Yu, et al. Adaptive layer splitting for wireless llm inference in edge computing: A model-based reinforcement learning approach. arXiv preprint arXiv:2406.02616, 2024

2024 arXiv
[204]

Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference

Shengyuan Ye, Jiangsu Du, Liekang Zeng, et al. Galaxy: A resource-efficient collaborative edge ai system for in-situ transformer inference. In Proc. IEEE INFOCOM, pages 1001–1010, 2024

2024
[205]

Efficient edge inference by selective query

Anil Kag, Igor Fedorov, Aditya Gangrade, et al. Efficient edge inference by selective query. In Proc. 11th Int. Conf. Learn. Represent., ICLR, 2023

2023
[206]

Weak-to-strong search: align large language models via searching over small language models

Zhanhui Zhou, Zhixuan Liu, Jie Liu, et al. Weak-to-strong search: align large language models via searching over small language models. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2025

2025
[207]

Flash: Latent-aware semi-autoregressive speculative decoding for multimodal tasks

Zihua Wang, Ruibo Li, Haozhe Du, et al. Flash: Latent-aware semi-autoregressive speculative decoding for multimodal tasks. arXiv preprint arXiv:2505.12728, 2025

2025
[208]

On speculative decoding for multimodal large language models

Mukul Gagrani, Raghavv Goel, Wonseok Jeon, et al. On speculative decoding for multimodal large language models. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops, CVPRW , pages 8285–8289, 2024

2024
[209]

Accelerating greedy coordinate gradient and general prompt optimization via probe sampling

Yiran Zhao, Wenyue Zheng, Tianle Cai, et al. Accelerating greedy coordinate gradient and general prompt optimization via probe sampling. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2025

2025
[210]

Seed: Accelerating reasoning tree construction via scheduled speculative decoding

Zhenglin Wang, Jialong Wu, Yilong Lai, et al. Seed: Accelerating reasoning tree construction via scheduled speculative decoding. In Proc. Int. Conf. Comput. Linguist., COLING , pages 4920–4937, 2025

2025
[211]

Speculative rag: Enhancing retrieval augmented generation through drafting

Zilong Wang, Zifeng Wang, Long Le, et al. Speculative rag: Enhancing retrieval augmented generation through drafting. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[212]

Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling

Wenda Xu, Rujun Han, Zifeng Wang, et al. Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[213]

Accelerated diffusion models via speculative sampling

Valentin De Bortoli, Alexandre Galashov, Arthur Gretton, et al. Accelerated diffusion models via speculative sampling. In Proc. Int. Conf. Mach. Learn., ICML , 2025

2025
[214]

Banditspec: Adaptive speculative decoding via bandit algorithms

Cunxiao Du Yunlong Hou, Fengzhuo Zhang et al. Banditspec: Adaptive speculative decoding via bandit algorithms. In Proc. Int. Conf. Mach. Learn., ICML, 2025

2025
[215]

Fast large language model collaborative decoding via speculation

Jiale Fu, Yuchu Jiang, Junkai Chen, et al. Fast large language model collaborative decoding via speculation. In Proc. Int. Conf. Mach. Learn., ICML, 2025. , Vol. 1, No. 1, Article . Publication date: September 2025. 34 • Senyao Li, Haozhao Wang, Wenchao Xu, Rui Zhang, Song Guo...

2025
[216]

Diffusion models are secretly exchangeable: Parallelizing ddpms via autospeculation

Hengyuan Hu, Aniket Das, Dorsa Sadigh, et al. Diffusion models are secretly exchangeable: Parallelizing ddpms via autospeculation. In Proc. Int. Conf. Mach. Learn., ICML , 2025

2025
[217]

Spectr: fast speculative decoding via optimal transport

Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, et al. Spectr: fast speculative decoding via optimal transport. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2023

2023
[218]

Glide with a cape: a low-hassle method to accelerate speculative decoding

Cunxiao Du, Jing Jiang, Xu Yuanchen, et al. Glide with a cape: a low-hassle method to accelerate speculative decoding. In Proc. Int. Conf. Mach. Learn., ICML , 2024

2024
[219]

Bass: Batched attention-optimized speculative sampling

Haifeng Qian, Sujan Kumar Gonugondla, Sungsoo Ha, et al. Bass: Batched attention-optimized speculative sampling. In Findings Assoc. Comput. Linguist., ACL, pages 8214–8224, 2024

2024
[220]

Swift: On-the-fly self-speculative decoding for llm inference acceleration

Heming Xia, Yongqi Li, Jun Zhang, et al. Swift: On-the-fly self-speculative decoding for llm inference acceleration. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[221]

Efficient inference for large language model-based generative recommendation

Xinyu Lin, Chaoqun Yang, Wenjie Wang, et al. Efficient inference for large language model-based generative recommendation. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[222]

An efficient private gpt never autoregressively decodes

Zhengyi Li, Yue Guan, Kang Yang, et al. An efficient private gpt never autoregressively decodes. In Proc. Int. Conf. Mach. Learn., ICML , 2025

2025
[223]

Speculate, then collaborate: Fusing knowledge of language models during decoding

Ziyao Wang, Muneeza Azmat, Ang Li, et al. Speculate, then collaborate: Fusing knowledge of language models during decoding. In Proc. Int. Conf. Mach. Learn., ICML , 2025

2025
[224]

Autodroid: Llm-powered task automation in android

Hao Wen, Yuanchun Li, Guohong Liu, et al. Autodroid: Llm-powered task automation in android. In Proc. 30th Annu. Int. Conf. Mobile Comput. Netw., MobiCom, page 543–557, 2024

2024
[225]

Biomistral: A collection of open-source pretrained large language models for medical domains

Yanis Labrak, Adrien Bazoge, Emmanuel Morin, et al. Biomistral: A collection of open-source pretrained large language models for medical domains. In Findings Assoc. Comput. Linguist., ACL , pages 5848–5864, 2024

2024
[226]

Language models meet world models: Embodied experiences enhance language models

Jiannan Xiang, Tianhua Tao, Yi Gu, et al. Language models meet world models: Embodied experiences enhance language models. Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 36:75392–75412, 2023

2023
[227]

Towards an on-device agent for text rewriting

Yun Zhu, Yinxiao Liu, Felix Stahlberg, et al. Towards an on-device agent for text rewriting. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , pages 2535–2552, 2024

2024
[228]

Chateval: Towards better LLM-based evaluators through multi-agent debate

Chi-Min Chan, Weize Chen, Yusheng Su, et al. Chateval: Towards better LLM-based evaluators through multi-agent debate. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[229]

Dialogue summarization with mixture of experts based on large language models

Yuanhe Tian, Fei Xia, and Yan Song. Dialogue summarization with mixture of experts based on large language models. In Proc. 62nd Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 7143–7155, 2024

2024
[230]

A real-world webagent with planning, long context understanding, and program synthesis

Izzeddin Gur, Hiroki Furuta, Austin V Huang, et al. A real-world webagent with planning, long context understanding, and program synthesis. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024
[231]

Accelerating inference of retrieval-augmented generation via sparse context selection

Yun Zhu, Jia-Chen Gu, Caitlin Sikora, et al. Accelerating inference of retrieval-augmented generation via sparse context selection. In Proc. 13th Int. Conf. Learn. Represent., ICLR , 2025

2025
[232]

Minillm: Large language models on consumer gpus

Volodymyr Kuleshov. Minillm: Large language models on consumer gpus. https://github.com/kuleshov/minillm, 2023

2023
[235]

Qwen2.5 technical report

Qwen, An Yang, Baosong Yang, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2025

2025 arXiv
[236]

Openelm: An efficient language model family with open training and inference framework

Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, et al. Openelm: An efficient language model family with open training and inference framework. arXiv preprint arXiv:2404.14619, 2024

2024 arXiv
[237]

GLM: General language model pretraining with autoregressive blank infilling

Zhengxiao Du, Yujie Qian, Xiao Liu, et al. GLM: General language model pretraining with autoregressive blank infilling. In Proc. 60th Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 320–335, 2022

2022
[238]

Galactica: A large language model for science

Ross Taylor, Marcin Kardas, Guillem Cucurull, et al. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022

2022 arXiv
[239]

Gemma 2: Improving open language models at a practical size, 2024

Gemma Team, Morgane Riviere, Shreya Pathak, et al. Gemma 2: Improving open language models at a practical size, 2024

2024
[240]

Gpt-4 technical report

OpenAI, Josh Achiam, Steven Adler, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2024

2024 arXiv
[241]

Chatglm: A family of large language models from glm-130b to glm-4 all tools

Team GLM. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024

2024 arXiv
[242]

Top ten technology trends of damo academy, 2022

Alibaba Damo Academy. Top ten technology trends of damo academy, 2022

2022
[243]

The future of ai is hybrid; part i: Unlocking the generative ai future with on-device and hybrid ai

Qualcomm. The future of ai is hybrid; part i: Unlocking the generative ai future with on-device and hybrid ai. https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part- 1-Unlocking-the-generative-AI-future-with-on-device...

2024
[244]

Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding

Heming Xia, Zhe Yang, Qingxiu Dong, et al. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. In Findings Assoc. Comput. Linguist., ACL , pages 7655–7671, 2024

2024
[245]

The future of ai is hybrid; part ii: Qualcomm is uniquely positioned to scale hybrid ai

Qualcomm. The future of ai is hybrid; part ii: Qualcomm is uniquely positioned to scale hybrid ai. https://www.qualcomm.com/content/ dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-2-Qualcomm-is-uniquely-positioned-to- , Vol. 1, No. 1, Article ...

2025
[246]

Deploying foundation model powered agent services: A survey

Wenchao Xu, Jinyu Chen, Peirong Zheng, et al. Deploying foundation model powered agent services: A survey. IEEE Commun. Surv. Tutor., pages 1–1, 2025

2025
[247]

Remoterag: A privacy-preserving llm cloud rag service.arXiv preprint arXiv:2412.12775, 2024

Yihang Cheng, Lan Zhang, Junyang Wang, et al. Remoterag: A privacy-preserving llm cloud rag service.arXiv preprint arXiv:2412.12775, 2024

2024 arXiv
[248]

Unleashing the power of edge-cloud generative ai in mobile networks: A survey of aigc services

Minrui Xu, Hongyang Du, Dusit Niyato, et al. Unleashing the power of edge-cloud generative ai in mobile networks: A survey of aigc services. IEEE Commun. Surv. Tutor., 26(2):1127–1170, 2024

2024
[249]

Ai-enhanced cloud-edge-terminal collaborative network: Survey, applications, and future directions

Huixian Gu, Liqiang Zhao, Zhu Han, et al. Ai-enhanced cloud-edge-terminal collaborative network: Survey, applications, and future directions. IEEE Commun. Surv. Tutor., 26(2):1322–1385, 2024

2024
[250]

Edge cloud offloading algorithms: Issues, methods, and perspectives

Jianyu Wang, Jianli Pan, Flavio Esposito, et al. Edge cloud offloading algorithms: Issues, methods, and perspectives. ACM Comput. Surv., 52(1):2:1–2:23, 2019

2019
[251]

Litemoe: Customizing on-device llm serving via proxy submodel tuning

Yan Zhuang, Zhenzhe Zheng, Fan Wu, et al. Litemoe: Customizing on-device llm serving via proxy submodel tuning. In Proc. 22nd ACM Conf. Embedded Networked Sensor Syst., SenSys , page 521–534, 2024

2024
[252]

Early-exit deep neural network - A comprehensive survey

Haseena Rahmath P, Vishal Srivastava, Kuldeep Chaurasia, et al. Early-exit deep neural network - A comprehensive survey. ACM Comput. Surv., 57(3):75:1–75:37, 2025

2025
[253]

Jie Ou, Yueming Chen, and Prof. Tian. Lossless acceleration of large language model via adaptive n-gram parallel decoding. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Papers), NAACL , pages 10–22, 2024

2024
[254]

Ecoagent: An efficient edge-cloud collaborative multi-agent framework for mobile automation

Biao Yi, Xavier Hu, Yurun Chen, et al. Ecoagent: An efficient edge-cloud collaborative multi-agent framework for mobile automation. arXiv preprint arXiv:2505.05440, 2025

2025
[255]

Large-scale multi-agent learning-based cloud–edge collaborative distributed pv data compression and information aggregation for multimodal network in power systems

Junhao Feng, Boyang Huang, Xiaodong Zhou, et al. Large-scale multi-agent learning-based cloud–edge collaborative distributed pv data compression and information aggregation for multimodal network in power systems. IEEE Trans. Consum. Electron. , 71(1):30–40, 2025

2025
[256]

Eclm: Efficient edge-cloud collaborative learning with continuous environment adaptation

Yan Zhuang, Zhenzhe Zheng, Yunfeng Shao, et al. Eclm: Efficient edge-cloud collaborative learning with continuous environment adaptation. arXiv preprint arXiv:2311.11083, 2023

2023 arXiv
[257]

A cloud-edge collaboration solution for distribution network reconfiguration using multi-agent deep reinforcement learning

Hongjun Gao, Renjun Wang, Shuaijia He, et al. A cloud-edge collaboration solution for distribution network reconfiguration using multi-agent deep reinforcement learning. IEEE Trans. Power Syst., 39(2):3867–3879, 2024

2024
[258]

Fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding

Sangmin Bae, Jongwoo Ko, Hwanjun Song, et al. Fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 5910–5924, 2023

2023
[259]

Investigating acceleration of llama inference by enabling intermediate layer decoding via instruction tuning with ’lite’

Neeraj Varshney, Agneet Chatterjee, Mihir Parmar, et al. Investigating acceleration of llama inference by enabling intermediate layer decoding via instruction tuning with ’lite’. In Proc. 2024 Conf. North Am. Chap. Assoc. Comput. Linguist.: Hum. Lang. Technol. (Vol. 1: Long Pa...

2024
[260]

Finding the SWEET spot: Analysis and improvement of adaptive inference in low resource settings

Daniel Rotem, Michael Hassid, Jonathan Mamou, et al. Finding the SWEET spot: Analysis and improvement of adaptive inference in low resource settings. In Proc. 61st Annu. Meet. Assoc. Comput. Linguist. (ACL), Vol. 1, ACL , pages 14836–14851, 2023

2023
[261]

Ee-llm: Large-scale training and inference of early-exit large language models with 3d parallelism

Yanxi Chen, Xuchen Pan, Yaliang Li, et al. Ee-llm: Large-scale training and inference of early-exit large language models with 3d parallelism. In Proc. Int. Conf. Mach. Learn., ICML , 2024

2024
[262]

Edge-cloud polarization and collaboration: A comprehensive survey for ai

Jiangchao Yao, Shengyu Zhang, Yang Yao, et al. Edge-cloud polarization and collaboration: A comprehensive survey for ai. IEEE Trans. on Knowl. and Data Eng. , 35(7):6866–6886, 2023

2023
[263]

Intelligent request strategy design in recommender system

Xufeng Qian, Yue Xu, Fuyu Lv, et al. Intelligent request strategy design in recommender system. In Proc. ACM SIGKDD Conf. Knowl. Discov. Data Min., KDD, page 3772–3782, 2022

2022
[264]

Walle: An end-to-end,general-purpose, and large-scale production system for device-cloud collaborative machine learning

Chengfei Lv, Chaoyue Niu, Renjie Gu, et al. Walle: An end-to-end,general-purpose, and large-scale production system for device-cloud collaborative machine learning. In 16th USENIX Symp. Oper. Syst. Des. Implementation, OSDI , pages 249–265, 2022

2022
[265]

Luoxi models

LuoXi Team. Luoxi models. https://github.com/luoxi-model/luoxi_models, 2024. Accessed: 2025-06-28

2024
[266]

Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection

Yuhang Liu, Pengxiang Li, Zishu Wei, et al. Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection. arXiv preprint arXiv:2501.04575, 2025

2025 arXiv
[268]

A unified approach to routing and cascading for llms

Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for llms. arXiv preprint arXiv:2410.10347, 2024

2024 arXiv
[269]

Llm bandit: Cost-efficient llm generation via preference-conditioned dynamic routing

Yang Li. Llm bandit: Cost-efficient llm generation via preference-conditioned dynamic routing. arXiv preprint arXiv:2502.02743, 2025

2025 arXiv
[270]

Dynamic ensemble reasoning for llm experts

Jinwu Hu, Yufeng Wang, Shuhai Zhang, et al. Dynamic ensemble reasoning for llm experts. arXiv preprint arXiv:2412.07448, 2024

2024
[271]

Blending is all you need: Cheaper, better alternative to trillion-parameters llm

Xiaoding Lu, Zongyi Liu, Adian Liusie, et al. Blending is all you need: Cheaper, better alternative to trillion-parameters llm. arXiv preprint arXiv:2401.02994, 2024

2024 arXiv
[272]

The moe-empowered edge llms deployment: Architecture, challenges, and opportunities

Ning Li, Song Guo, Tuo Zhang, et al. The moe-empowered edge llms deployment: Architecture, challenges, and opportunities. arXiv preprint arXiv:2502.08381, 2025. , Vol. 1, No. 1, Article . Publication date: September 2025. 36 • Senyao Li, Haozhao Wang, Wenchao Xu, Rui Zhang, So...

2025 arXiv
[273]

Slimrag: Retrieval without graphs via entity-aware context selection

Jiale Zhang, Jiaxiang Chen, Zhucong Li, et al. Slimrag: Retrieval without graphs via entity-aware context selection. arXiv preprint arXiv:2506.17288, 2025

2025 arXiv
[274]

Arag: Agentic retrieval augmented generation for personalized recommendation

Reza Yousefi Maragheh, Pratheek Vadla, Priyank Gupta, et al. Arag: Agentic retrieval augmented generation for personalized recommendation. arXiv preprint arXiv:2506.21931, 2025

2025 arXiv
[275]

Algorithmics and complexity of cost-driven task offloading with submodular optimization in edge-cloud environments

Longkun Guo, Jiawei Lin, Xuanming Xu, et al. Algorithmics and complexity of cost-driven task offloading with submodular optimization in edge-cloud environments. arXiv preprint arXiv:2411.15687, 2024

2024 arXiv
[276]

Cross-attention speculative decoding

Wei Zhong, Manasa Bharadwaj, Yixiao Wang, et al. Cross-attention speculative decoding. arXiv preprint arXiv:2505.24544, 2025

2025
[277]

Graft: Integrating the domain knowledge via efficient parameter synergy for mllms

Yang Dai, Jianxiang An, Tianwei Lin, et al. Graft: Integrating the domain knowledge via efficient parameter synergy for mllms. arXiv preprint arXiv:2506.23940, 2025

2025 arXiv
[278]

Federated learning-based data collaboration method for enhancing edge cloud ai system security using large language models

Huaiying Luo and Cheng Ji. Federated learning-based data collaboration method for enhancing edge cloud ai system security using large language models. arXiv preprint arXiv:2506.18087, 2025

2025 arXiv
[279]

Cloud–edge–end collaborative federated learning: Enhancing model accuracy and privacy in non-iid environments

Ling Li, Lidong Zhu, and Weibang Li. Cloud–edge–end collaborative federated learning: Enhancing model accuracy and privacy in non-iid environments. Sensors, 24(24), 2024

2024
[280]

Parameter-efficient domain knowledge integration from multiple sources for biomedical pre-trained language models

Qiuhao Lu, Dejing Dou, and Thien Huu Nguyen. Parameter-efficient domain knowledge integration from multiple sources for biomedical pre-trained language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proc. Conf. Empir. Methods Nat...

2021
[281]

Acknowledge: Acquired knowledge representation by small language model without pre-training

Sourav Das, Sanjay Chatterji, and Imon Mukherjee. Acknowledge: Acquired knowledge representation by small language model without pre-training. In Proc. 1st Workshop Towards Knowl. Lang. Models, KnowLLM , pages 83–95, 2024

2024
[282]

Knowslm: A framework for evaluation of small language models for knowledge augmentation and humanised conversations

Chitranshu Harbola and Anupam Purwar. Knowslm: A framework for evaluation of small language models for knowledge augmentation and humanised conversations. arXiv preprint arXiv:2504.04569, 2025

2025 arXiv
[283]

A comprehensive survey in llm(-agent) full stack safety: Data, training and deployment

Kun Wang, Guibin Zhang, Zhenhong Zhou, et al. A comprehensive survey in llm(-agent) full stack safety: Data, training and deployment. arXiv preprint arXiv:2504.15585, 2025

2025 arXiv
[284]

Hawkeye:efficient reasoning with model collaboration

Jianshu She, Zhuohao Li, Zhemin Huang, et al. Hawkeye:efficient reasoning with model collaboration. arXiv preprint arXiv:2504.00424, 2025

2025 arXiv
[285]

Vita-1.5: Towards gpt-4o level real-time vision and speech interaction

Chaoyou Fu, Haojia Lin, Xiong Wang, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction. arXiv preprint arXiv:2501.01957, 2025

2025 arXiv
[286]

Bounding and filling: A fast and flexible framework for image captioning

Zheng Ma, Changxin Wang, Bo Huang, et al. Bounding and filling: A fast and flexible framework for image captioning. In Nat. Lang. Process. Chin. Comput., NLPCC , pages 469–481, 2023

2023
[287]

Thoughtful things: Building human-centric smart devices with small language models

Evan King, Haoxiang Yu, Sahil Vartak, et al. Thoughtful things: Building human-centric smart devices with small language models. arXiv preprint arXiv:2405.03821, 2024

2024 arXiv
[288]

Harnessing multiple large language models: A survey on llm ensemble

Zhijun Chen, Jingzheng Li, Pengpeng Chen, et al. Harnessing multiple large language models: A survey on llm ensemble. arXiv preprint arXiv:2502.18036, 2025

2025 arXiv
[289]

Deepseek-v3 technical report

DeepSeek-AI, Aixin Liu, Bei Feng, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2025

2025 arXiv
[290]

Luan, et al

Fahao Chen, Peng Li, Tom H. Luan, et al. Spin: Accelerating large language model inference with heterogeneous speculative models. In Proc. IEEE INFOCOM, pages 1–10, 2025

2025
[291]

Fast and cost-effective speculative edge-cloud decoding with early exits, 2025

Yeshwanth Venkatesha, Souvik Kundu, and Priyadarshini Panda. Fast and cost-effective speculative edge-cloud decoding with early exits, 2025

2025
[292]

UPOA: A user preference based latency and energy aware intelligent offloading approach for cloud-edge systems

Jingling Yuan, Yao Xiang, Yuhui Deng, et al. UPOA: A user preference based latency and energy aware intelligent offloading approach for cloud-edge systems. IEEE Trans. Cloud Comput. , 11(2):2188–2203, 2023

2023
[293]

Differentially private low-rank adaptation of large language model using federated learning

Xiao-Yang Liu, Rongyi Zhu, Daochen Zha, et al. Differentially private low-rank adaptation of large language model using federated learning. ACM Trans. Manag. Inf. Syst. , 16(2):1–24, 2025

2025
[294]

QLLMS: quantization-adaptive LLM scheduling for partially informed edge serving systems

Miao Hu, Qi He, and Di Wu. QLLMS: quantization-adaptive LLM scheduling for partially informed edge serving systems. In Proc. IEEE INFOCOM, pages 1–10. IEEE, 2025

2025
[295]

Enhancing stability and resource efficiency in llm training for edge-assisted mobile systems

Chang Liu and Jun Zhao. Enhancing stability and resource efficiency in llm training for edge-assisted mobile systems. IEEE Trans. Mob. Comput., pages 1–18, 2025. , Vol. 1, No. 1, Article . Publication date: September 2025

2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [162]

A survey on collaborative mechanisms between large and small language models

Yi Chen, JiaHao Zhao, and HaoHao Han. A survey on collaborative mechanisms between large and small language models. arXiv preprint arXiv:2505.07460, 2025

arXiv 2025

[2] [163]

Collaborative learning of on-device small model and cloud-based large model: Advances and future directions

Chaoyue Niu, Yucheng Ding, Junhui Lu, et al. Collaborative learning of on-device small model and cloud-based large model: Advances and future directions. arXiv preprint arXiv:2504.15300, 2025

arXiv 2025

[3] [84]

Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding

Kaiyan Zhang, Jianyu Wang, Ning Ding, et al. Fast and slow generating: An empirical study on large and small language models collaborative decoding. arXiv preprint arXiv:2406.12295, 2024

work page Pith review arXiv 2024

[4] [12]

Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices

Ruslan Svirschevski, Avner May, Zhuoming Chen, et al. Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices. arXiv preprint arXiv:2406.02532, 2024

arXiv 2024

[5] [124]

Cloud-edge collaborative intelligent inference based on distributed neural networks in power distribution networks

Hao Luo, Hui Tian, Peng Zhang, et al. Cloud-edge collaborative intelligent inference based on distributed neural networks in power distribution networks. In Int. Conf. Space-Air-Ground Comput., SAGC, pages 129–136, 2021

2021

[6] [40]

Guiding reasoning in small language models with llm assistance

Yujin Kim, Euiin Yi, Minu Kim, et al. Guiding reasoning in small language models with llm assistance. arXiv preprint arXiv:2504.09923, 2025

arXiv 2025

[7] [42]

Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices

Ruslan Svirschevski, Avner May, Zhuoming Chen, et al. Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices. In Proc. Int. Conf. Neural Inf. Process. Syst., NeurIPS , 2024

2024

[8] [233]

Minillm: Knowledge distillation of large language models

Yuxian Gu, Li Dong, Furu Wei, et al. Minillm: Knowledge distillation of large language models. In Proc. 12th Int. Conf. Learn. Represent., ICLR, 2024

2024

[9] [157]

Tinybert: Distilling BERT for natural language understanding

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al. Tinybert: Distilling BERT for natural language understanding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 4163–4174, 2020

2020

[10] [234]

TinyBERT: Distilling BERT for natural language understanding

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al. TinyBERT: Distilling BERT for natural language understanding. In Proc. Conf. Empir. Methods Nat. Lang. Process., EMNLP , pages 4163–4174, 2020

2020

[11] [267]

Hybrid edge-ai framework for intelligent mobile applications: Leveraging large language models for on-device contextual assistance and code-aware automation

Liao Hu. Hybrid edge-ai framework for intelligent mobile applications: Leveraging large language models for on-device contextual assistance and code-aware automation. J. Ind. Eng. Appl. Sci. , 3(3):10–22, 2025

2025

[12] [111]

Large language model cascades with mixture of thought representations for cost-efficient reasoning

Murong Yue, Jie Zhao, Min Zhang, et al. Large language model cascades with mixture of thought representations for cost-efficient reasoning. In Proc. 12th Int. Conf. Learn. Represent., ICLR , 2024

2024

[13] [117]

Edgeshard: Efficient llm inference via collaborative edge computing

Mingjin Zhang, Xiaoming Shen, Jiannong Cao, et al. Edgeshard: Efficient llm inference via collaborative edge computing. IEEE Internet Things J., 12(10):13119–13131, 2025

2025

[14] [116]

Pipeedge: Pipeline parallelism for large-scale model inference on heterogeneous edge devices

Yang Hu, Connor Imes, Xuanang Zhao, et al. Pipeedge: Pipeline parallelism for large-scale model inference on heterogeneous edge devices. In 25th Euromicro Conf. Digit. Syst. Des., DSD , pages 298–307, 2022

2022