REVIEW 5 cited by
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present EE-LLM, a framework for large-scale training and inference of early-exit large language models (LLMs). While recent works have shown preliminary evidence for the efficacy of early exiting in accelerating LLM inference, EE-LLM makes a foundational step towards scaling up early-exit LLMs by supporting their training and inference with massive 3D parallelism. Built upon Megatron-LM, EE-LLM implements a variety of algorithmic innovations and performance optimizations tailored to early exiting, including a lightweight method that facilitates backpropagation for the early-exit training objective with pipeline parallelism, techniques of leveraging idle resources in the original pipeline schedule for computation related to early-exit layers, and two approaches of early-exit inference that are compatible with KV caching for autoregressive generation. Our analytical and empirical study shows that EE-LLM achieves great training efficiency with negligible computational overhead compared to standard LLM training, as well as outstanding inference speedup without compromising output quality. To facilitate further research and adoption, we release EE-LLM at https://github.com/pan-x-c/EE-LLM.
Forward citations
Cited by 5 Pith papers
-
Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
Hidden-state probes can flag doomed LLM-agent episodes from the first round, and a recall-calibrated cascade of abort gates cuts generated tokens by up to 60% while preserving a chosen success-recall target.
-
SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning
A small tool-free MLLM plus answer-separability gating bypasses agentic tool loops for many queries, yielding 1.1–3.35× speedup with preserved or higher accuracy.
-
Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspective
No single LLM quantization method dominates performance, energy, and quality; the best choice depends on task, request length, load, parallelism, and GPU type.
-
Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits
Edge-cloud speculative decoding runs faster when early exits in the server model let the client pre-draft the next candidate tokens before final verification is complete.
-
SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling
SkipGPT learns per-token, per-module routers that dynamically skip attention and MLP blocks, then uses LoRA to recover most of the original model's quality.
Discussion (0). Sign in to comment.