Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models

Ahmed E. Hassan; Dayi Lin; Kirill Vasilevski

REVIEW 4 major objections 5 minor 1 cited by

Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RAR claims that a router can cut expensive-model calls by 50.2% while retaining 90.5% of response quality by teaching the weak model with strong-model-generated guides.

desk verdict Plausible closed-loop routing idea with honest limitations, but the headline cost and quality numbers are not yet supported because guide calls may be excluded and quality is measured by alignment, not correctness. read the letter →

arxiv 2411.09837 v2 pith:TZDTF25Z submitted 2024-11-14 cs.LG cs.AIcs.MA

Kirill Vasilevski , Dayi Lin , Ahmed E. Hassan This is my paper · ORCID

classification cs.LGcs.AIcs.MA

keywords LLMroutinglayeredfoundationmodelsin-contextlearningcontinualchain-of-thoughtguidescost-qualitytrade-offsemanticsimilarityMMLUbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Real-time Adaptive Routing (RAR), a router for software that uses a cheaper, weaker language model by default and a stronger, more expensive one as fallback. The claim is that the router can improve over time by having the strong model supply step-by-step reasoning guides that the weak model uses in its prompt, so that requests originally sent to the strong model can later be handled by the weak model. On subsets of the MMLU multiple-choice benchmark, RAR reports routing 50.2% fewer requests to the strong model while preserving 90.5% of the response quality of an oracle static router, with comparable results on two additional domains and a second strong model. The practical point is that the expensive model can continuously train the cheap model during normal operation, lowering inference cost without retraining weights.

What carries the argument

The mechanism is a learned loop of shadow inference, semantic comparison, and memory reuse. When the static router would pick the strong model, RAR also generates a weak-model response in the background; a vector-embedding similarity check against the strong model's response decides whether the weak model can serve this type of request alone. If not, RAR asks the strong model for a guide, a step-by-step reasoning hint that does not contain the answer, and runs the weak model again with the guide prepended to the request. Successful request-guide pairs are stored in a vector database indexed by cosine similarity of sentence embeddings, and the matching threshold acts as an exploration-versus-exploitation dial that controls how eagerly the system reuses old guides instead of requesting new ones.

What would settle it

On the same MMLU subsets, score RAR's responses against the benchmark's ground-truth answers instead of against the strong model's responses: if the requests RAR routes to the weak model have substantially lower accuracy than the strong model's accuracy, or if aligned responses often disagree with the correct multiple-choice answer, the quality-retention claim is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that routing decisions do not have to stay fixed after deployment. Each time the static router sends a request to the strong model, RAR shadows the request by also asking the weak model to answer it; if the weak model's response is semantically aligned with the strong model's, the request is saved as a skill, and future similar requests are routed directly to the weak model. If the weak model fails alone, RAR obtains a guide from the strong model and retries the weak model with that guide in context, saving successful request-guide pairs for reuse. In the reported experiments this adaptation reduces strong-model use by 50.2% and retains 90.5% of the aligned-response count of an oracle static router, with statistical significance at $p<0.001$; the stored guides show within-domain reuse and a smaller amount of cross-domain transfer. The paper defines an aligned response as one semantically similar to the strong model's response, not necessarily the ground-truth-correct one.

Load-bearing premise

The load-bearing premise is that a response counts as good when it is semantically similar to what the stronger model would have said; if that similarity does not track correctness, the 90.5% quality-retention figure does not show real quality.

Editorial extensions

If this is right

A deployed layered LLM service can expect roughly half of the requests that a static router would send to the expensive model to be absorbed by the cheap model after enough guide accumulation, at a small alignment-cost penalty.
The benefit grows with request repetition and topical clustering: similar or recurring user requests become progressively cheaper because they are served from skill and guide memory.
Guide reuse is not per-question memorization; the paper reports intra-domain generalization, with one guide helping multiple related questions, and small cross-domain transfer of 6-7% over an unguided weak model.
Because the guide memory is filled from the stronger FM's own reasoning, the routing loop converts expensive-model calls into reusable assets, so long-running deployments get more efficient rather than staying at the static router's cost level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because RAR's quality metric is alignment with the strong model's output, the system inherits the strong model's errors: a confident but wrong strong-model answer would be treated as the reference, so the reported 90.5% shows fidelity to the strong model, not objective accuracy.
A natural next experiment, flagged by the paper's own validity discussion, is to test RAR on open-ended generation with an LLM judge for semantic comparison; the multiple-choice format is a constrained case that may overstate transferability.
The similarity threshold is effectively a cost-quality control: raising it forces more fresh strong-model guides, lowering it favors reuse and accepts less similar requests.
In edge deployments the same loop acts as a personalization mechanism: guides accumulate from the user's own request history, so the on-device weak model should improve for that user's recurring needs while keeping data on the device.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Short version: this is a plausible and honest paper about a useful idea, but the two headline numbers ('50.2% fewer expensive calls,' '90.5% quality retention') are not actually nailed down by the experiments as written. I would send it to review, but only with a clear expectation of major revision.

What's new: the loop of static router, shadow inference on the weaker model, storing requests and generated guides in a vector memory, and using that memory to route future requests to the weak model. Each piece exists, but the closed-loop adaptation for routing is a reasonable combination. The strongest evidence is Figure 7: with each stage, more successful guides come from memory rather than fresh strong-model calls. That shows intra-domain generalization is real, not just memorization.

Soft spots, in order of size.

First, the cost metric is ambiguous. Section IV-A3 says they record 'the number of times stronger FM is used,' but never defines whether guide-generation calls are included. In Case 2, a guide can be generated by the stronger FM on top of the response call. Figure 7 tracks guide source separately from memory, which strongly suggests the 'Used stronger FM' count in Figure 4 counts only direct response calls. If that's true, a request that needs both a response and a fresh guide costs two strong-model calls, and the 50.2% reduction over the oracle (which makes one call per difficult request) could disappear or reverse. This is the load-bearing issue.

Second, 'quality' is alignment with the stronger FM's response, not correctness. MMLU has ground truth; they could report accuracy, but don't. Their alignment metric is honest about being a proxy, but then the 90.5% number is '90.5% of alignment,' not '90.5% of quality' as most readers will interpret it.

Third, the oracle static router is privileged information, so it's a conservative baseline—good for a first test, but no real router (e.g., RouteLLM) is compared, so the practical claim 'improves on existing routing' is not directly demonstrated. The threshold on similarity is a free parameter and not sensitivity-analyzed.

The paper is clearly written and the threats-to-validity section is honest about the repeated-samples concern. No signs of data fabrication. It's just that the headline claims outrun the measurements.

Recommendation: engage with it for peer review, but make the authors define the cost metric exactly, count guide calls, and report accuracy on the benchmark. If the cost claim survives that, the paper is a solid empirical contribution.

Best, [Your name]

Referee Report

4 major / 5 minor

Summary. The paper proposes Real-time Adaptive Routing (RAR), a continual-learning router for software powered by layered foundation models. When a static predictive router sends a request to a stronger, more expensive FM, RAR performs shadow inference with a weaker, cheaper FM: if the weak response is semantically similar to the strong response (an 'aligned response'), the request and any guide are stored in skill/guide memory; future similar requests are then served by the weak FM, optionally with a cached guide. The paper evaluates RAR on three filtered subsets of MMLU (professional law, moral scenarios, high-school psychology) using Mistral-7B as the weak model and GPT-4o or Llama-3-70B as the strong model. It reports a 50.2% reduction in strong-model calls while retaining 90.5% of response quality relative to an oracle static router (p<0.001), and reports evidence of intra-domain and some inter-domain guide generalization.

Significance. RAR addresses a practically relevant cost/quality trade-off, and the core idea - using strong-model step-by-step reasoning as reusable in-context guides to improve routing decisions over time - is plausible and interesting. The paper evaluates across three domains and two different strong models, and the intra-domain guide-reuse trend in Figure 7 is a useful empirical finding. However, the headline quality claim is measured by semantic similarity to the strong model's response rather than by correctness or user-facing quality, and the cost metric does not state whether guide-generation calls to the strong model are counted. These issues mean the headline 50.2% / 90.5% numbers are not yet established. The paper would be substantially strengthened by reporting MMLU accuracy and a complete accounting of all strong-model calls, including guide generation.

major comments (4)

[III-D, IV-A3, IV-B2] The central cost-reduction claim (50.2% fewer stronger-FM calls, Section IV-B2) is computed from 'the number of times stronger FM is used' (Section IV-A3), but the paper never states whether stronger-FM calls made to generate guides under Case 2 (Section III-D) are included. Since Figure 7 separately tracks 'guide generation with stronger FM' versus 'guide memory', it is plausible that Figure 4's 'Used stronger FM' counts only direct response calls and excludes guide-generation calls. If so, a request answered by the strong model that also requires a fresh guide incurs two strong-model calls under RAR, while the oracle static router incurs only one; adding these guide calls could materially reduce or even reverse the reported cost advantage. Please define the cost metric precisely and re-report the efficiency numbers with all stronger-FM calls, including guide generation, included.
[III-A, IV-B2] The paper claims to maintain '90.5% of response quality,' but the evaluation metric is the number of 'aligned responses,' defined in Section III-A as responses that are semantically similar to the stronger FM's output, not responses that are correct or high-quality. The authors themselves note that 'aligned response' differs from 'correct response.' Because RAR's objective (increase aligned responses) and the evaluation metric (number of aligned responses) are the same quantity, the retention figure is partly a restatement of the optimization target rather than independent evidence of quality. Since the evaluation uses MMLU, ground-truth answers are available; please report accuracy (or another user-oriented quality metric) for RAR and all baselines, and use that for the quality-retention claim.
[IV-A1, Figure 3] The datasets are not the full MMLU subsets named in the abstract. Figure 3 shows that all samples that the weaker FM can answer correctly are discarded, leaving only the 754 law, 359 psychology, and 675 moral-scenario failing samples. Consequently, the reported 50.2% and 90.5% figures apply only to a filtered, deliberately more difficult subset, and the abstract should not describe this as evaluation on 'different subsets of MMLU' without that qualification. Moreover, each stage reuses the same samples, so the cumulative aligned-response counts and the cost reduction over stages partly reflect cached solutions to repeated, identical requests; please also report performance on novel requests or on a single pass to separate memorization/caching from generalization.
[IV-A2, III-F] The main results depend on the similarity-score threshold (set to 0.2 in Section IV-A2), the retry period for Case 3 re-checking (Section III-D), and the number of experiment stages (Section IV-A3), but no sensitivity analysis is reported for any of these. The threshold directly controls the exploration/exploitation trade-off and defines which responses count as aligned, so the headline numbers should be shown over a range of thresholds (and, if feasible, over different retry periods and stage counts). Without such analysis, it is hard to know how robust the 50.2% / 90.5% result is to reasonable hyperparameter choices.

minor comments (5)

[Abstract, IV-B2] The abstract and Section IV-B2 report '50.2% reduction,' while the RQ1 summary in Section IV-B2 reports '50.1% reduction'; please make these numbers consistent or explain the difference.
[IV-B1] The 'oracle static router' baseline is constructed using privileged knowledge of the dataset (profiling determines which samples the weaker model can answer). This is a useful upper-bound baseline, but the paper should state explicitly that it is an oracle and not a realistic deployable router, so readers do not interpret the comparison as one against a practical static router.
[Table I] The caption of Table I says 'difference between the cumulative number of aligned responses vs. stronger FM (lower is better)' and reports values such as '15.0%' and '47,1%'. Please define the formula used, fix the comma in '47,1%', and state whether these are per-stage or cumulative percentages.
[References] Reference [26] and [27] are the same paper (Shnitzer et al., “Large Language Model Routing with Benchmark Datasets”) with duplicate entries; please cite it once.
[IV-A3] The statistical testing is described only as 'Chi-square test with 95% confidence interval' without stating which contingency tables were compared, what the comparison groups were, or the test statistics and degrees of freedom. Please report the actual test setup so the p<0.001 values are verifiable.

Circularity Check

1 steps flagged · score 6.0 of 10

RAR's headline quality-retention metric is the same statistic as its learning objective: 'aligned responses' are defined as semantic similarity to the stronger FM, and that same alignment signal is what populates the memory that the metric then counts.

self definitional [Section III-A (definition of aligned response); Section III-D (memory update rule); Section IV-A3 (evaluation metric); Section IV-B2 (headline result)]
""It is important to note that by aligned response, we define the case as a weaker FM generates a semantically-similar response to that of a stronger FM, which is different from generating the correct response given a request." ... "We evaluate the capability of the method by how many responses are aligned with the stronger FMs response." ... "Any time that a weaker FM generates an aligned response, the request and guide (if used) are recorded into a skill and guide memory.""

The headline 'maintaining 90.5% of response quality' (Section IV-B2) is measured by the number of aligned responses. But 'aligned response' is defined as semantic similarity to the stronger FM's response, and Section III-D shows that RAR's memory is populated only when the weaker FM produces exactly this kind of aligned response. Thus the adaptation signal, the memory-construction rule, and the evaluation metric are all the same statistic. The 90.5% retention figure is therefore a measure of how well RAR optimizes its own definition of success, not an independent validation of response quality. The paper even acknowledges alignment differs from correctness, so the 'quality' language re-labels the optimization target.

full rationale

RAR is an online system whose core loop is: run shadow inference, compare weaker and stronger outputs by semantic similarity, store the request/guide when they are similar, and later route similar requests to the weaker model. Evaluating the system by the count of such semantically-similar outputs is a coherent way to measure the method, but it is not an external check of the central quality claim. The score reflects that the quality-retention claim is partially circular because 'response quality' is defined as the metric being optimized. The cost-reduction comparison to the oracle static router is empirical and not circular (modulo the separate ambiguity of whether guide-generation calls are counted, which is a reporting/validity issue rather than a circularity issue). No load-bearing self-citations or imported uniqueness theorems appear in the paper; citations to prior work are background or external benchmarks.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The paper's claims rest on the semantic-alignment proxy, the similarity-threshold choice, and the omission of guide-generation cost from the count of strong-model usage. No new physical or theoretical entities are introduced.

free parameters (3)

similarity score threshold = 0.2 (RQ1), 0.1 (inter-domain)
Set based on the median sample-wise cosine similarity (0.442) on the professional law subset, the same data used for evaluation; controls how similar a stored request must be for guide reuse.
retry period for Case 3 re-check = unspecified (tuned hyperparameter)
The paper states Case 3 requests retry shadow inference after 'a certain period of time (a tuned hyperparameter)' with no value given.
number of experiment stages = 5
Defines how many passes over the dataset are used to populate memory; outcomes may depend on this choice.

assumptions (6)

domain assumption A response that is semantically similar to the stronger FM's response has the same quality as the stronger FM's response.
Central quality metric; introduced in Section III-A and used throughout Section IV-B.
domain assumption Embedding-based cosine similarity between requests is sufficient to identify requests that can reuse the same guide.
Skill and guide memory uses all-MiniLM-L12-v2 embeddings and cosine similarity (Sections III-F, IV-A2).
domain assumption Step-by-step reasoning from a stronger FM is useful to a weaker FM via in-context learning.
Core hypothesis stated in Section III: the stronger FM can provide insightful guides that the weaker FM can use.
domain assumption A static predictive router provides a reasonable initial routing decision in real deployment.
Section III-C describes using an initial static router, but the evaluation does not actually include a trained static router.
domain assumption The selected MMLU subsets (samples the weak FM fails) represent hard requests in a real workload.
Section IV-A1 pre-filters to 754 law, 359 psychology, 675 moral-scenario samples.
domain assumption Five random shuffles of the sample order are enough to characterize sequence-dependence.
Section IV-A3 uses five permutations; no variance or statistical detail is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models." pith.science (2026). https://pith.science/paper/TZDTF25Z

@misc{pith2026241109837,
  author       = {Pith},
  title        = {Pith review of: Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZDTF25Z}},
  note         = {Machine review of arXiv:2411.09837}
}

read the original abstract

To balance the quality and inference cost of a Foundation Model (FM, such as large language models (LLMs)) powered software, people often opt to train a routing model that routes requests to FMs with different sizes and capabilities. Existing routing models rely on learning the optimal routing decision from carefully curated data, require complex computations to be updated, and do not consider the potential evolution of weaker FMs. In this paper, we propose Real-time Adaptive Routing (RAR), an approach to continuously adapt FM routing decisions while using guided in-context learning to enhance the capabilities of weaker FM. The goal is to reduce reliance on stronger, more expensive FMs. We evaluate our approach on different subsets of the popular MMLU benchmark. Over time, our approach routes 50.2% fewer requests to computationally expensive models while maintaining around 90.5% of the general response quality. In addition, the guides generated from stronger models have shown intra-domain generalization and led to a better quality of responses compared to an equivalent approach with a standalone weaker FM.

Figures

Figures reproduced from arXiv: 2411.09837 by the authors.

**Figure 2.** Overview of RAR procedure and shadow inference, described in Section III; IQ - incoming request [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** B. RQ1: Is it possible to reduce reliance on stronger FM meanwhile maintaining similar capability levels? 1) Approach We compared our approach to the following methods: standalone stronger FM, standalone weaker FM, weaker FM with zero-shot CoT [32] reasoning, and an oracle static router system. Chain-of-Thought approach is included since the use of reasoning in prompts has been shown to improve the quality of genera… view at source ↗

Figures from the paper (5 more)

**Figure 3.** Figure 3: Overview of data selection and single stage in the experiment process. First, samples from select MMLU domains are [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

**Figure 4.** Figure 4: Cumulative number of aligned responses and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Cumulative number of aligned responses (blue, higher [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Cumulative number of aligned responses (blue, higher [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Cumulative number of aligned guided responses per [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Adaptive Minds: Empowering Agents with LoRA-as-Tools
cs.AI 2025-10 reject novelty 4.0 of 10

Adaptive Minds makes a base LLM select LoRA adapters as tools per query; the 5-adapter demo gets 100% routing on 25 queries, while the abstract's 30-adapter/nine-family numbers are unsupported.

Reference graph

Works this paper leans on

34 extracted references · 16 canonical work pages · cited by 1 Pith paper

[3]

Abhimanyu Dubey et al.The Llama 3 Herd of Models

work page
[4]

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah Abdin et al. “Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone”. In: ArXivabs/2404.14219 (2024)

arXiv 2024
[5]

Tom Gunter et al.Apple Intelligence Foundation Lan- guage Models. 2024. arXiv: 2407 . 21075[cs.AI]. URL: https://arxiv.org/abs/2407.21075

arXiv 2024
[6]

Graph of Thoughts: Solving Elab- orate Problems with Large Language Models

Maciej Besta et al. “Graph of Thoughts: Solving Elab- orate Problems with Large Language Models”. In:Pro- ceedings of the AAAI Conference on Artificial Intelli- gence38.16 (Mar. 2024)

work page 2024
[7]

Skills-in-Context Prompting: Unlock- ing Compositionality in Large Language Models

Jiaao Chen et al. “Skills-in-Context Prompting: Unlock- ing Compositionality in Large Language Models”. In: ArXivabs/2308.00304 (2023)

arXiv 2023
[8]

Fru- galML: how to use ML prediction APIs more accurately and cheaply

Lingjiao Chen, Matei Zaharia, and James Zou. “Fru- galML: how to use ML prediction APIs more accurately and cheaply”. In:Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20. Vancouver, BC, Canada: Curran Associates Inc., 2020

work page 2020
[9]

Lingjiao Chen, Matei Zaharia, and James Zou.Frugal- GPT: How to Use Large Language Models While Re- ducing Cost and Improving Performance. 2023. arXiv: 2305.05176[cs.LG].URL: https://arxiv.org/abs/2305. 05176

arXiv 2023
[10]

Dujian Ding et al.Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing. 2024. arXiv: 2404.14618 [cs.LG].URL: https://arxiv.org/abs/2404.14618

arXiv 2024

Show all 34 references

[11]

Hassan et al.Towards AI-Native Software En- gineering (SE 3.0): A Vision and a Challenge Roadmap

Ahmed E. Hassan et al.Towards AI-Native Software En- gineering (SE 3.0): A Vision and a Challenge Roadmap
[12]

org/abs/2410.06107

arXiv: 2410.06107[cs.SE].URL: https://arxiv. org/abs/2410.06107
[13]

Measuring Massive Multitask Language Understanding

Dan Hendrycks et al. “Measuring Massive Multitask Language Understanding”. In:ArXivabs/2009.03300 (2020)

2020 arXiv
[14]

Qitian Jason Hu et al.RouterBench: A Benchmark for Multi-LLM Routing System. 2024. arXiv: 2403.12031 [cs.LG].URL: https://arxiv.org/abs/2403.12031

2024 arXiv
[15]

Jiang et al.Mistral 7B

Albert Q. Jiang et al.Mistral 7B. 2023. arXiv: 2310. 06825[cs.CL].URL: https : / / arxiv. org / abs / 2310 . 06825

2023
[16]

LLM- Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. “LLM- Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion”. In:Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Ed. by Anna Rogers,...

2023
[17]

Scaling Laws for Neural Language Models

Jared Kaplan et al. “Scaling Laws for Neural Language Models”. In:ArXivabs/2001.08361 (2020)

2020 arXiv
[18]

Retrieval-augmented generation for knowledge-intensive NLP tasks

Patrick Lewis et al. “Retrieval-augmented generation for knowledge-intensive NLP tasks”. In:Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20. Vancouver, BC, Canada: Curran Associates Inc., 2020

2020
[19]

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Zechun Liu et al. “MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases”. In:ArXivabs/2402.14905 (2024)

2024 arXiv
[20]

AutoMix: Automatically Mixing Language Models

Aman Madaan et al. “AutoMix: Automatically Mixing Language Models”. In:ArXivabs/2310.12963 (2023)

2023 arXiv
[21]

CLIN: A Contin- ually Learning Language Agent for Rapid Task Adap- tation and Generalization

Bodhisattwa Prasad Majumder et al. “CLIN: A Contin- ually Learning Language Agent for Rapid Task Adap- tation and Generalization”. In:ArXivabs/2310.10134 (2023)

2023 arXiv
[22]

The Chi-square test of indepen- dence

Mary L. McHugh. “The Chi-square test of indepen- dence”. In:Biochemia Medica(2013)

2013
[23]

https://ai.meta.com/ blog/llama- 3- 2- connect- 2024- vision- edge- mobile- devices/ [Accessed: (Sept

Meta.Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. https://ai.meta.com/ blog/llama- 3- 2- connect- 2024- vision- edge- mobile- devices/ [Accessed: (Sept. 25, 2024)]. 2024

2024
[24]

Isaac Ong et al.RouteLLM: Learning to Route LLMs with Preference Data. 2024. arXiv: 2406 . 18665 [cs.LG].URL: https://arxiv.org/abs/2406.18665

2024 arXiv
[25]

https://openai.com/index/hello- gpt-4o/

OpenAI.Hello GPT-4o. https://openai.com/index/hello- gpt-4o/. [Accessed 30-09-2024]. [25]Qdrant - Vector Database — qdrant.tech. https://qdrant. tech/. [Accessed 02-10-2024]

2024
[27]

Tal Shnitzer et al.Large Language Model Routing with Benchmark Datasets. 2023. arXiv: 2309 . 15789 [cs.CL].URL: https://arxiv.org/abs/2309.15789

2023 arXiv
[28]

https://huggingface.co/sentence-transformers/all- MiniLM-L12-v2

Sentence Transformers.all-MiniLM-L12-v2 - Hugging Face. https://huggingface.co/sentence-transformers/all- MiniLM-L12-v2. [Accessed 01-10-2024]

2024
[29]

V oyager: An Open-Ended Em- bodied Agent with Large Language Models

Guanzhi Wang et al. “V oyager: An Open-Ended Em- bodied Agent with Large Language Models”. In:Trans. Mach. Learn. Res.2024 (2023)

2023
[30]

A Comprehensive Survey of Continual Learning: Theory, Method and Application

L. Wang et al. “A Comprehensive Survey of Continual Learning: Theory, Method and Application”. In:IEEE Transactions on Pattern Analysis & Machine In- telligence46.08 (Aug. 2024)

2024
[31]

Tabi: An Efficient Multi-Level Inference System for Large Language Models

Yiding Wang et al. “Tabi: An Efficient Multi-Level Inference System for Large Language Models”. In: Proceedings of the Eighteenth European Conference on Computer Systems. EuroSys ’23. Rome, Italy: Associa- tion for Computing Machinery, 2023

2023
[32]

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei et al. “Chain-of-thought prompting elicits reasoning in large language models”. In:Proceedings of the 36th International Conference on Neural Infor- mation Processing Systems. NIPS ’22. New Orleans, LA, USA: Curran Associates Inc., 2024

2024
[33]

Tree of Thoughts: Deliberate Prob- lem Solving with Large Language Models

Shunyu Yao et al. “Tree of Thoughts: Deliberate Prob- lem Solving with Large Language Models”. In:ArXiv abs/2305.10601 (2023)

2023 arXiv
[34]

A Survey of Large Language Models

Wayne Xin Zhao et al. “A Survey of Large Language Models”. In:ArXivabs/2303.18223 (2023)

2023 arXiv
[35]

Lianmin Zheng et al.Judging LLM-as-a-Judge with MT- Bench and Chatbot Arena. 2023. arXiv: 2306 . 05685 [cs.CL].URL: https://arxiv.org/abs/2306.05685

2023 arXiv
[36]

Judging LLM-as-a-judge with MT-bench and Chatbot Arena

Lianmin Zheng et al. “Judging LLM-as-a-judge with MT-bench and Chatbot Arena”. In:Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23. New Orleans, LA, USA: Curran Associates Inc., 2024

2024
[2024]

org/abs/2407.21783

arXiv: 2407.21783[cs.AI].URL: https://arxiv. org/abs/2407.21783

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [3]

Abhimanyu Dubey et al.The Llama 3 Herd of Models

work page

[2] [4]

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah Abdin et al. “Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone”. In: ArXivabs/2404.14219 (2024)

arXiv 2024

[3] [5]

Tom Gunter et al.Apple Intelligence Foundation Lan- guage Models. 2024. arXiv: 2407 . 21075[cs.AI]. URL: https://arxiv.org/abs/2407.21075

arXiv 2024

[4] [6]

Graph of Thoughts: Solving Elab- orate Problems with Large Language Models

Maciej Besta et al. “Graph of Thoughts: Solving Elab- orate Problems with Large Language Models”. In:Pro- ceedings of the AAAI Conference on Artificial Intelli- gence38.16 (Mar. 2024)

work page 2024

[5] [7]

Skills-in-Context Prompting: Unlock- ing Compositionality in Large Language Models

Jiaao Chen et al. “Skills-in-Context Prompting: Unlock- ing Compositionality in Large Language Models”. In: ArXivabs/2308.00304 (2023)

arXiv 2023

[6] [8]

Fru- galML: how to use ML prediction APIs more accurately and cheaply

Lingjiao Chen, Matei Zaharia, and James Zou. “Fru- galML: how to use ML prediction APIs more accurately and cheaply”. In:Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20. Vancouver, BC, Canada: Curran Associates Inc., 2020

work page 2020

[7] [9]

Lingjiao Chen, Matei Zaharia, and James Zou.Frugal- GPT: How to Use Large Language Models While Re- ducing Cost and Improving Performance. 2023. arXiv: 2305.05176[cs.LG].URL: https://arxiv.org/abs/2305. 05176

arXiv 2023

[8] [10]

Dujian Ding et al.Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing. 2024. arXiv: 2404.14618 [cs.LG].URL: https://arxiv.org/abs/2404.14618

arXiv 2024