REVIEW 16 cited by
Liger Kernel: Efficient Triton Kernels for LLM Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Training Large Language Models (LLMs) efficiently at scale presents a formidable challenge, driven by their ever-increasing computational demands and the need for enhanced performance. In this work, we introduce Liger-Kernel, an open-sourced set of Triton kernels developed specifically for LLM training. With kernel optimization techniques like kernel operation fusing and input chunking, our kernels achieve on average a 20% increase in training throughput and a 60% reduction in GPU memory usage for popular LLMs compared to HuggingFace implementations. In addition, Liger-Kernel is designed with modularity, accessibility, and adaptability in mind, catering to both casual and expert users. Comprehensive benchmarks and integration tests are built in to ensure compatibility, performance, correctness, and convergence across diverse computing environments and model architectures. The source code is available under a permissive license at: github.com/linkedin/Liger-Kernel.
Forward citations
Cited by 16 Pith papers
-
RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
A single transformer-based model trained once on synthetic scenes can render novel triangle meshes with global illumination, without per-scene fine-tuning.
-
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
SlideFormer uses layer-sliding async offloading, pre-allocated heterogeneous memory, and fused Triton kernels to fine-tune 123B+ models on one RTX 4090 with 1.4–6.3× higher throughput and roughly half the memory of pr...
-
The Appeal and Reality of Recycling LoRAs with Adaptive Merging
Adaptive merging of recycled LoRAs gives little benefit over training a target-task LoRA, and randomly initialized LoRAs work as well as real ones once the target LoRA is in the pool.
-
LRM-1B: Towards Large Routing Model
A 1B-parameter routing model beats existing multi-task neural solvers on synthetic VRP benchmarks, and the authors fit power-law scaling curves for model size, trajectories, and compute.
-
Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences
ALST combines tiled loss and MLP compute, Ulysses-style sequence parallelism for Hugging Face GQA and MQA models, and activation-checkpoint CPU offload to train Llama-8B at up to 500K tokens on one H100 and 15M tokens...
-
Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
Enhancing a model's long-context ability before supervised fine-tuning improves math reasoning accuracy, including on short-input benchmarks.
-
SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
SelfCite uses context-ablation probability differences as a self-supervised reward to improve LLM sentence-level citations, raising LongBench-Cite citation F1 from 73.8 to 79.1.
-
Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
A Thai 70B model trained with an SFT-plus-DARE-merge recipe matches DeepSeek R1 on reasoning benchmarks while retaining most Thai language quality.
-
Typhoon T1: An Open Thai Reasoning Model
Structured long-thinking SFT turns a 3B Thai instruct model into a reasoning model that improves on several English benchmarks and can think in Thai, with a fully open recipe.
-
SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
An Ascend-NPU training stack reaches 34.22% MFU on DeepSeek-V4-Pro, and a solver-verified CPT+SFT recipe raises OR benchmark averages to 71.81% (Flash) and 77.33% (Pro).
-
RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment
A fully open 518M Arabic-specialized LLM, built by vocabulary injection and standard post-training on Qwen2.5-0.5B, beats same-class multilingual baselines and ships at 398 MB quantized.
-
Docopilot: Improving Multimodal Models for Document-Level Understanding
A new academic-paper dataset and a retrieval-free fine-tuned InternVL2 model improve multi-page document QA accuracy and latency on several benchmarks.
-
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
TransAct V2 adds lifelong user sequences and a next-action loss to Pinterest's CTR model, reporting online gains of +6.35% repin volume and -12.80% hide volume.
-
BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation
BenchHub is an automatically categorized, customizable LLM benchmark suite covering 303K questions across 38 benchmarks in English and Korean.
-
100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
100-LongBench and LongScore evaluate LLMs at controlled context lengths and rank models by relative performance drop from a short-context baseline, not by raw accuracy.
-
The Hitchhiker's Guide to Agentic AI: From Foundations to Systems
A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.
Discussion (0). Continue with ORCID to comment.