REVIEW 4 cited by
Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Due to the cost-prohibitive nature of training Large Language Models (LLMs), fine-tuning has emerged as an attractive alternative for specializing LLMs for specific tasks using limited compute resources in a cost-effective manner. In this paper, we characterize sparse Mixture of Experts (MoE) based LLM fine-tuning to understand their accuracy and runtime performance on a single GPU. Our evaluation provides unique insights into the training efficacy of sparse and dense versions of MoE models, as well as their runtime characteristics, including maximum batch size, execution time breakdown, end-to-end throughput, GPU hardware utilization, and load distribution. Our study identifies the optimization of the MoE layer as crucial for further improving the performance of LLM fine-tuning. Using our profiling results, we also develop and validate an analytical model to estimate the cost of LLM fine-tuning on the cloud. This model, based on parameters of the model and GPU architecture, estimates LLM throughput and the cost of training, aiding practitioners in industry and academia to budget the cost of fine-tuning a specific model.
Forward citations
Cited by 4 Pith papers
-
Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs
On consumer GPUs, LoRA+ gives the best energy-focused fine-tuning score in 19 of 24 small-model task configurations, while QLoRA wins the memory-focused score when peak VRAM is the binding constraint.
-
Edge-Deployable LLM Fine-Tuning on a Single GPU for Telecom Network Troubleshooting
Single-GPU Unsloth SFT+RFT of 7–8B models is feasible for telecom edge troubleshooting once sequence length, GPU utilization, and chat-template handling are profiled per architecture.
-
A Word is Worth 4-bit: Efficient Log Parsing with Binary Coded Decimal Recognition
A 312k-parameter character-level log parser using 4-bit BCD encoding matches LLM-based parser accuracy at up to 20x higher throughput.
-
Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes
Closed-source LLMs can produce systems engineering artifact text that scores nearly identically to a human expert benchmark on MAUVE text similarity, but expert review of the best-scoring outputs reveals three failure...
Discussion (0). Continue with ORCID to comment.