REVIEW 14 cited by
MLPerf Inference Benchmark
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
MLPerf Inference Benchmark
read the original abstract
Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and five orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. In this paper, we present our benchmarking method for evaluating ML inference systems. Driven by more than 30 organizations as well as more than 200 ML engineers and practitioners, MLPerf prescribes a set of rules and best practices to ensure comparability across systems with wildly differing architectures. The first call for submissions garnered more than 600 reproducible inference-performance measurements from 14 organizations, representing over 30 systems that showcase a wide range of capabilities. The submissions attest to the benchmark's flexibility and adaptability.
Forward citations
Cited by 14 Pith papers
-
FAIR+S: A validation study of a framework for sustainable research data and software
FAIR+S extends FAIR with sustainability metrics and is validated via expert survey confirming importance but revealing awareness gaps in green practices.
-
Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining
Multilingual pretraining develops translation in two phases: early copying driven by surface similarities, followed by generalizing mechanisms while copying is refined.
-
SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving
SLO-Guard improves tuning budget consistency for SLO-constrained LLM serving by handling crashes explicitly and using a two-phase feasible-first exploration plus exploitation strategy.
-
A Switch-Centric In-Network Architecture for Accelerating LLM Inference in Shared-Memory Network
SCIN uses an in-switch accelerator for direct memory access and 8-bit in-network quantization during All-Reduce, delivering up to 8.7x faster small-message reduction and 1.74x TTFT speedup on LLaMA-2 models.
-
Edge-Inference Governors Need Memory-Clock State
EMC-blind GPU-only latency fits miss 25–28% of tight deadlines on Jetson Orin; an EMC-aware two-cell refit holds misses ≤1.3% under a 2% QoS budget and selects a budget-feasible clock.
-
The xPU-athalon: Quantifying the Competition of AI Acceleration
Quantitative benchmarks across recent AI accelerators reveal that optimal hardware choice varies with workload parameters and that several platforms incur substantially higher idle power than GPUs.
-
Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
Watt Counts supplies over 5,000 energy measurements across 50 LLMs and 10 GPUs and shows that hardware-aware selection can reduce server-scenario energy use by up to 70 percent with little effect on user experience.
-
Scalable Synthesis of distributed LLM workloads through Symbolic Tensor Graphs
STAGE synthesizes high-fidelity Chakra-format execution graphs for distributed LLM workloads from symbolic tensor definitions, validated against real 128-GPU H100 traces and scaled to 32K GPUs.
-
MatCreatioNN: Machine learning-guided computational discovery of photocatalysts for environmental applications
A ML funnel selects Zn- and Cr-based MOFs scoring 1.2-1.7x above PCN-224(Zr), but the comparison uses the same fitness score that selected them.
-
Edge-Inference Governors Need Memory-Clock State
EMC state is required in latency models for edge inference governors; EMC-blind CPU/GPU fits miss 25-28% deadlines while EMC-aware refits limit misses to 1.3% and identify feasible energy points across vision and LLM ...
-
HAFM: Hierarchical Autoregressive Foundation Model for Music Accompaniment Generation
A three-stage hierarchical AR model with dual-rate HuBERT/EnCodec tokens improves vocal-conditioned accompaniment generation, reaching FAD 1.71 and 51.5% preference vs ground truth on MUSDB18.
-
HAFM: Hierarchical Autoregressive Foundation Model for Music Accompaniment Generation
HAFM uses a hierarchical autoregressive model with dual-rate HuBERT and EnCodec tokens to generate coherent instrumental music from vocals, achieving FAD 2.08 on MUSDB18 while matching prior systems with fewer parameters.
-
Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
Nvidia achieves 1.6x throughput with NVFP4 but hits a VRAM wall for 70B+ models, while Apple UMA enables linear scaling to 80B at 4-bit with up to 23x better energy efficiency.
-
Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining
SLO-Guard, a crash-aware two-phase autotuner for vLLM serving, achieves no best-latency improvement over random search but demonstrates more consistent budget allocation across 150 trials on Qwen2-1.5B/A100.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.