REVIEW 6 cited by
Eagle: Efficient Training-Free Router for Multi-LLM Inference
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need by dynamically choosing the most suitable model for a given query based on task requirements and budget constraints. However, existing routers face challenges in scalability and real-time adaptation, particularly in high-volume online environments. We present Eagle, a novel LLM routing approach that combines global and local ELO ranking modules to overcome these limitations. By evaluating both general and specialized LLM abilities, Eagle provides a scalable, training-free solution that enhances model selection quality while reducing computational overhead. Our experiments across multiple datasets show Eagle consistently outperforms baseline methods, with improvements of up to 23.52 percent in Area Under Curve (AUC) scores. Moreover, Eagle demonstrates remarkable efficiency, requiring only 1/20 of baseline methods' time for initialization and 100 to 200 times faster incremental updates in online scenarios, making it well-suited for dynamic, high-volume online serving environments.
Forward citations
Cited by 6 Pith papers
-
Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts
A DRL router using graph attention state abstraction and QoS-aware rewards improves average QoS by up to 35.78% over four baselines in simulated edge LLM routing.
-
Orchestration for Domain-specific Edge-Cloud Language Models
ECO-LLM jointly selects query processing, retrieval, and model components per query, cutting cost by 60% and latency up to 6x versus model routing in edge-cloud tests.
-
Universal Model Routing for Efficient LLM Inference
UniRoute represents each language model by its error rates on a few prompt clusters, letting a router choose among models it has never seen during training.
-
Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalization
Uncertainty-based routing thresholds for small-to-large LLM offloading can be bootstrapped from a calibration set built on other datasets, because confidence distributions depend more on the small model and uncertaint...
-
Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes
Statistical classifiers built on LLM activation norms and coordinates match or beat trained MLP heads on coarse intent routing and resist camouflage better, while MLPs win on fine-grained subfield distinctions.
-
Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systems
A survey that classifies LLM routing strategies into pre-generation and post-generation approaches and four implementation families, framed as a performance-cost optimization problem.
Discussion (0). Continue with ORCID to comment.