REVIEW 6 cited by
Learning to Decode Collaboratively with Multiple Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent variable. By optimizing the marginal likelihood of a training set under our latent variable model, the base LLM automatically learns when to generate itself and when to call on one of the ``assistant'' language models to generate, all without direct supervision. Token-level collaboration during decoding allows for a fusion of each model's expertise in a manner tailored to the specific task at hand. Our collaborative decoding is especially useful in cross-domain settings where a generalist base LLM learns to invoke domain expert models. On instruction-following, domain-specific QA, and reasoning tasks, we show that the performance of the joint system exceeds that of the individual models. Through qualitative analysis of the learned latent decisions, we show models trained with our method exhibit several interesting collaboration patterns, e.g., template-filling. Our code is available at https://github.com/clinicalml/co-llm.
Forward citations
Cited by 6 Pith papers
-
Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning
Token-wise minimum and base-relative consensus over per-source fine-tunes suppress hidden and explicit poisoning that union training and weight averaging retain.
-
PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
PyroDash lets a small model decide mid-generation to hand off once to a frozen LLM, achieving 64.04% accuracy (+6.36 points vs LLM) at lambda=0.05 and 96.4% lower estimated cost at lambda=0.6.
-
Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units
MCSU-based vocabulary alignment plus distance-based dynamic selection (DDS) lets several LLMs vote token-by-token, beating single models and prior ensemble baselines on multiple reasoning benchmarks without training.
-
What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
FoReaL-Decoding lets a strong reasoning model generate the first few tokens of each sentence and a weaker model complete the sentence, cutting theoretical FLOPs by 30-55% while retaining 86-100% of accuracy on four ma...
-
Toward Edge General Intelligence with Multiple-Large Language Model (Multi-LLM): Architecture, Trust, and Orchestration
A survey of multi-LLM systems in edge computing, covering architectures, enabling technologies, trust mechanisms, applications, and open datasets for edge general intelligence.
-
Scalable, Symbiotic, AI and Non-AI Agent Based Parallel Discrete Event Simulations
PDES orchestration of small language models with non-AI verifier agents raises accuracy on four toy tasks from about 23 percent to 68 percent, with the verifiers supplying most of the correctness.
Discussion (0). Sign in to comment.