SNLP achieves up to 2.58x wall-clock speedup on 0.5B Transformers via architecture-specific Newton corrections (IDN/HCN) that enable layer-parallel inference while preserving perplexity in milder settings.
Taming the titans: A survey of efficient LLM inference serving.arXiv preprint arXiv:2504.19720,
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Presents a recency/frequency adaptive KV caching approach that achieves up to 10.8% higher hit rate and 12.6% lower TTFT compared to vLLM on synthetic workloads.
citing papers explorer
-
SNLP: Layer-Parallel Inference via Structured Newton Corrections
SNLP achieves up to 2.58x wall-clock speedup on 0.5B Transformers via architecture-specific Newton corrections (IDN/HCN) that enable layer-parallel inference while preserving perplexity in milder settings.
-
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Presents a recency/frequency adaptive KV caching approach that achieves up to 10.8% higher hit rate and 12.6% lower TTFT compared to vLLM on synthetic workloads.