Pith. sign in

REVIEW 4 major objections 5 minor 94 references

PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PREBA shows that CPU-side data preprocessing, not the GPU, is the main bottleneck in MIG-based AI inference servers, and that an FPGA data processing unit plus a MIG-aware dynamic batching system removes it.

desk verdict Real implementation and a clear MIG preprocessing characterization, but the missing accuracy check for FPGA-preprocessed inputs has to be answered before the 3.7x claim is fully like-for-like. read the letter →

arxiv 2411.19114 v1 pith:HDMMZNYM submitted 2024-11-28 cs.DC cs.AIcs.ARcs.LG

Gwangoo Yeo , Jiin Kim , Yujeong Choi , Minsoo Rhu This is my paper · ORCID
classification cs.DCcs.AIcs.ARcs.LG
keywords Multi-InstanceGPUAIinferenceservingFPGAdatapreprocessingdynamicbatchingtaillatencyutilizationMIGschedulinghardware/softwareco-design
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NVIDIA's Multi-Instance GPU (MIG) splits one large GPU into several smaller virtual GPUs (vGPUs), each capable of running an independent inference server. This paper claims that the resulting servers are not GPU-bound but preprocessing-bound: every activated vGPU multiplies the CPU-side work of decoding, resizing, and spectrogram computation, and the CPU cannot keep up. PREBA is the proposed fix, a hardware/software co-design that moves all data preprocessing onto an FPGA-based data processing unit and pairs it with a batching system whose parameters are chosen from per-model, per-vGPU profiling. In a real A100 system with six workloads, PREBA reports a 3.7x throughput gain, a 3.4x tail-latency reduction, a 3.5x energy-efficiency gain, and a 3.0x cost-efficiency gain over a CPU-preprocessing baseline. If these numbers hold, MIG becomes a practical way to serve latency-critical AI at high utilization.

What carries the argument

The load-bearing object is the batch knee, $\mathit{Batch_{knee}}$: the largest batch size before a vGPU's tail latency starts spiking with little throughput gain. The DPU's computing units (CUs), each a chain of functional units for decode-resize-crop-normalize or resample-mel-spectrogram-normalize, are the hardware mechanism that makes small batches cheap; the audio path splits into two CU types so the normalize stage (which needs all samples) does not serialize independent requests. The software half is a dynamic batching system that profiles throughput-versus-latency curves per model and MIG configuration, bucketizes variable audio lengths into queues, and computes $\mathit{Time_{queue}}$ analytically as $\mathit{Time_{knee}}$ divided by the vGPU count.

What would settle it

Compare end-to-end accuracy for the six workloads under CPU and DPU preprocessing on identical inputs: top-1 accuracy for MobileNet, SqueezeNet, and Swin-Transformer; word error rate for CitriNet and the two Conformers. A material accuracy gap (for example, more than about 0.5 points) would mean the measured 3.7x throughput does not translate into equally correct inference.

Watch

Extended reading notes

Core claim

The paper's central claim is that in a MIG inference server the scarce resource is not GPU compute but CPU preprocessing throughput. With seven vGPUs active, preprocessing demand scales sevenfold while CPU utilization saturates near 90%, causing a 75.6% end-to-end throughput drop in the authors' characterization. PREBA removes that bottleneck with an FPGA DPU whose computing units are deliberately optimized for single-input latency (rather than batch throughput), so requests are preprocessed immediately on arrival and the downstream batching stage has free choice of batch size. On top of that, the dynamic batching system sets the maximum batch size at the 'knee' of the tail-latency curve ($\mathit{Batch_{knee}}$), a point that is smaller for fine-grained vGPUs, and sets the queueing window as $\mathit{Time_{knee}}$ divided by the number of vGPUs. The measured result is that PREBA sustains 91.6% of an oracle 'Ideal' system's throughput on five of six models, while the CPU baseline loses 77.2% on average.

Load-bearing premise

The speedups are measured without verifying that the FPGA's JPEG, resize, and Mel-spectrogram outputs are numerically equivalent to the CPU's OpenCV/Librosa outputs, so model accuracy could shift even though throughput and latency improve.

Editorial extensions

If this is right

  • A MIG server with PREBA can reach over 91% of an oracle system's throughput, so AIaaS providers can partition a single A100 into seven vGPUs without paying the preprocessing tax.
  • Because $\mathit{Batch_{knee}}$ depends on vGPU size, model, and input length, the profiling step must be repeated per configuration; the paper provides a low-cost recipe for doing so.
  • Variable-length audio workloads should be batched by length bucket rather than by arrival count, which is what PREBA's per-bucket queues do.
  • Energy efficiency rises because the DPU displaces CPU cycles, even though GPU power consumption increases with higher utilization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If FPGA and CPU preprocessing are numerically identical, the same offload recipe could apply to tokenization and other light text preprocessing, but this paper does not test that claim.
  • The DPU→CPU→GPU data path adds tens of microseconds per request; at very high request rates or with multiple DPUs on one PCIe root complex, a direct DPU-to-GPU path (P2P) would likely be needed, which the paper notes but does not implement.
  • The $\mathit{Time_{queue}} = \mathit{Time_{knee}} / \#\text{vGPUs}$ rule assumes roughly even traffic across queues; bursty or skewed arrivals may require an adaptive controller rather than the fixed analytical formula.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies NVIDIA MIG-based AI inference servers and argues that CPU-side data preprocessing becomes a critical bottleneck as the GPU is partitioned into many vGPUs. To address this, PREBA offloads image and audio preprocessing to an FPGA-based DPU and adds a MIG-aware dynamic batching system that chooses batch size and queueing time from profiled latency/throughput curves. The system is implemented on an A100 with an Alveo U55C FPGA and evaluated on six vision and audio models. The reported results are an average 3.7x throughput improvement, 3.4x tail-latency reduction, 3.5x energy-efficiency improvement, and 3.0x cost-efficiency improvement over a CPU-preprocessing baseline, with an ablation study separating the DPU and dynamic batching contributions.

Significance. If the reported results are valid, PREBA addresses a real and increasingly important problem: MIG makes CPU preprocessing a bottleneck, and prior work on MIG inference has largely ignored the preprocessing stage. The paper's strengths are that it is built end-to-end on commodity hardware, uses six diverse workloads, includes a clear 'Ideal' oracle as an upper bound, and provides an ablation study that separates the hardware and software contributions. The main weakness is that the paper never demonstrates that the FPGA-preprocessed inputs preserve model accuracy relative to the CPU baseline, which is essential for interpreting the speedups as like-for-like. The dynamic batching heuristic also needs stronger justification. These issues are fixable with additional measurements and analysis, so the work is potentially a solid systems contribution if they are addressed.

major comments (4)
  1. [§4.2 and §6] The paper does not validate that FPGA-preprocessed inputs are functionally equivalent to the CPU baseline's OpenCV/Librosa outputs. Section 4.2 describes DPU units implemented with Vitis Vision and DSP libraries, and Section 6 reports only throughput, latency, power, and cost (Figures 17-21), with no accuracy comparison for any of the six workloads. JPEG decoding, resizing interpolation, FFT/Mel-filterbank, and mean/variance normalization can differ numerically between the CPU and FPGA implementations due to rounding, fixed-point precision, and algorithmic details, and for pretrained ImageNet and LibriSpeech models such differences can change predictions. The reported 3.7x throughput improvement is only meaningful if the DPU produces inputs that yield the same model accuracy as the CPU pipeline. Please add an accuracy comparison (e.g., top-1 accuracy on ILSVRC-2012 for vision models and word error rate on LibriSpeech for audio models) or a direct numerical equivalence check (e.g., maximum absolute or relative error against CPU reference outputs).
  2. [§6, Figures 17-20] All performance results are reported without run-to-run variance, confidence intervals, or the number of repetitions. Since the central claims are empirical (3.7x throughput, 3.4x tail latency, 3.5x energy efficiency, 3.0x cost efficiency), a single measurement per configuration leaves open the possibility that the improvements are within noise, especially for tail-latency metrics. Please report at least three to five independent runs per configuration and show mean and variance (or error bars) for the main throughput and latency figures.
  3. [§3.3 and §6.1] The baseline is a single CPU-based preprocessing implementation using OpenCV and Librosa on a 32-core EPYC server. The paper generalizes from this to claim that CPU-side preprocessing is the critical bottleneck in MIG inference, but it does not compare against a more optimized CPU pipeline (e.g., multi-threaded OpenCV, vectorized preprocessing, NVIDIA DALI, or GPU-side preprocessing). Section 3.3 reports that sustaining a 1g.5gb(7x) A100 would require 393 CPU cores for CitriNet, but this number depends on the efficiency of the specific CPU implementation. Without a state-of-the-art or at least a reasonably tuned CPU baseline, the 75.6% throughput drop and the 3.7x speedup may overstate the severity of the preprocessing bottleneck. Please add a comparison against a stronger CPU preprocessing baseline or explicitly scope the claim to the evaluated baseline.
  4. [§4.3, Figure 15] The analytical model for Time_queue rests on two assertions that are not fully justified. First, it claims that Time_knee is approximately constant at 35 ms regardless of audio input length, but Figure 15 shows this only for three audio models on 1g.5gb(7x), and the supporting data for other MIG configurations and models is not shown. Second, the paper sets Time_queue = Time_knee / (number of vGPUs) without a derivation or a sensitivity study; this division is presented as a heuristic, yet the batching section calls it an analytical model. The end-to-end results are encouraging, but the paper should either provide a short queueing-theoretic justification for the division by vGPU count or present a sensitivity analysis showing that the exact choice of this divisor does not materially affect the reported results.
minor comments (5)
  1. [Figure 21] The caption contains a typo: 'Tansformer' should be 'Transformer'.
  2. [Table 1] Table 1 reports FPGA resource utilization but not the operating frequency or the achieved preprocessing throughput per CU, so it is difficult to assess the efficiency of the DPU implementation. Please add the clock frequency and the per-unit throughput or latency numbers.
  3. [§4.3] The Time_queue formula is described in prose rather than as a numbered equation; please write it explicitly as an equation (e.g., Time_queue = Time_knee / V for V vGPUs) and relate it to the notation in Figure 15.
  4. [§5, Software] The text says the baseline uses 'Librosa' but reference [2] is 'LibrosaCpp'. Please clarify whether the baseline is the Python Librosa library or the C++ port, since numerical differences between these implementations could affect the claimed baseline preprocessing cost.
  5. [Section 6.1] The definition of 'Ideal' as a system with no preprocessing overhead is clear, but the statement that PREBA achieves '91.6% of the performance of Ideal for 5 out of the 6 studied models' would benefit from a per-model table so that the outlier (presumably Conformer large or CitriNet) is visible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PREBA's speedups are direct end-to-end measurements against a CPU-preprocessing baseline, and its batching hyperparameters are selected heuristically from profiling rather than used to reconstruct the reported results.

full rationale

The paper's central claims are empirical: the 3.7x throughput, 3.4x tail-latency reduction, 3.5x energy-efficiency, and 3.0x cost-efficiency improvements are measured end-to-end on real hardware (Figures 17-21) by comparing PREBA against a CPU-based OpenCV/Librosa preprocessing baseline, not derived from any fitted model or self-referential equation. The dynamic batching system selects Batch_max and Time_queue from profiled latency-throughput curves and a stated heuristic (Time_queue = Time_knee / number of vGPUs), but these profiled values are design hyperparameters, not quantities that are then algebraically identical to the reported speedups; the ablation study (Figure 22) separately attributes gains to the DPU and to dynamic batching via direct measurement. The 'Ideal' design is an oracle upper bound, and PREBA's closeness to it (91.6%) is a measured fraction rather than a construction. The authors do cite their own prior work (Lazy Batching, PREMA, PARIS/ELSA), but these citations are used for background, contrast, and related-work positioning, not as load-bearing support for PREBA's correctness or as an imported uniqueness theorem. The absence of an accuracy comparison between FPGA-preprocessed and CPU-preprocessed inputs is a real correctness risk, but it is not a circularity: it concerns whether the measured speedups preserve model quality, not whether the speedups reduce to the paper's own assumptions by definition.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four domain assumptions, most notably the unverified equivalence of FPGA and CPU preprocessing. Two batching-related parameters are chosen by hand or by heuristic division. No new physical entities are introduced; the FPGA DPU is a constructed system, not a postulated entity.

free parameters (3)
  • Audio bucket window = 2.5 seconds
    Chosen by hand to bucket variable-length audio inputs in the dynamic batching system; the choice affects how requests are grouped and how batch sizes are selected (Section 4.3, Figure 16).
  • Time_queue divisor = 1/7 for 1g.5gb(7x)
    Time_queue is set to Time_knee divided by the number of vGPUs. The 1/7 factor is a heuristic, not derived from a queueing model, and directly determines batching delay (Section 4.3).
  • Profiled Batch_knee thresholds = Model- and config-specific, e.g., 2 to 128
    Batch_max is set equal to the measured Batch_knee for each model, vGPU size, and input length (Section 4.3); these thresholds are calibration points rather than derived constants.
assumptions (4)
  • domain assumption MIG vGPU slices provide performance isolation with no cross-slice interference
    The paper treats each vGPU as an independent inference server throughout the characterization and evaluation (Sections 2.2 and 3); this is NVIDIA's architectural claim and is not independently verified in the paper.
  • domain assumption FPGA DPU preprocessing output is functionally equivalent to CPU OpenCV/Librosa output
    Offloading is presented as semantically preserving the preprocessing pipeline (Section 4.2), but no accuracy or output-equivalence measurement is reported for any workload.
  • ad hoc to paper Time_knee is roughly constant across audio input lengths and can be divided by vGPU count to choose Time_queue
    Empirical observation in Figure 15 (about 35 ms) and the heuristic Time_queue = Time_knee / N_vGPU in Section 4.3; the claim that this keeps all vGPUs busy is asserted rather than proven.
  • domain assumption Poisson-distributed query arrivals represent realistic inference traffic
    The evaluation uses Poisson arrivals following MLPerf recommendations (Section 5); this does not capture bursty or correlated production traffic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers." pith.science (2026). https://pith.science/paper/HDMMZNYM

@misc{pith2026241119114,
  author       = {Pith},
  title        = {Pith review of: PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HDMMZNYM}},
  note         = {Machine review of arXiv:2411.19114}
}
read the original abstract

NVIDIA's Multi-Instance GPU (MIG) is a feature that enables system designers to reconfigure one large GPU into multiple smaller GPU slices. This work characterizes this emerging GPU and evaluates its effectiveness in designing high-performance AI inference servers. Our study reveals that the data preprocessing stage of AI inference causes significant performance bottlenecks to MIG. To this end, we present PREBA, which is a hardware/software co-design targeting MIG inference servers. Our first proposition is an FPGA-based data preprocessing accelerator that unlocks the full potential of MIG with domain-specific acceleration of data preprocessing. The MIG inference server unleashed from preprocessing overheads is then augmented with our dynamic batching system that enables high-performance inference. PREBA is implemented end-to-end in real systems, providing a 3.7x improvement in throughput, 3.4x reduction in tail latency, 3.5x improvement in energy-efficiency, and 3.0x improvement in cost-efficiency.

Figures

Figures reproduced from arXiv: 2411.19114 by the authors.

Figure 1
Figure 1. Overview of NVIDIA’s GPU architecture. in tail latency, 3.5× improvement in energy-efficiency, and 3.0× improvement in cost-efficiency. 2 Background 2.1 AI Training vs. Inference in GPUs GPUs are throughput-optimized processors employing many￾core SIMD vectors with high-bandwidth memory (e.g., HBM [36] or GDDR [76]). Because AI training tasks employ large input batch sizes and exhibit throughput-hungry character￾ist… view at source ↗
Figure 2
Figure 2. MIG partitioning options in NVIDIA A100 GPU. AI inference server Model execution Inference engine GPU User request CPU Data preprocessing Batching Preprocessing engine Batch Batch [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. End-to-end AI inference pipeline. GPU hardware architecture [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Data preprocessing operations for (a) computer vision and (b) audio processing. Exploration of NVIDIA MIG. PARIS and ELSA [43] ex￾plored NVIDIA’s MIG architecture for AI inference, propos￾ing optimal MIG partitioning strategies as well as high￾performance scheduling po…
Figure 5
Figure 5. Figure 5: (Bar chart) Model execution throughput and (line chart) its GPU utilization when preprocessing is disabled. The x-axis shows the input batch size executed by a single vGPU. To separate out the effect of CPU-side data preprocessing on the GPU-side model execution, we ma…
Figure 7
Figure 7. Figure 7: Breakdown of average latency when 1g.5gb(7x) and 7g.40gb(1x) are configured with the appropriate 𝐵𝑎𝑡𝑐ℎ𝑚𝑎𝑥 value to sustain the same inference throughput. Data preprocessing stage is disabled in this experiment. GPU already reached close to its maximum possible through￾…
Figure 9
Figure 9. Figure 9: Throughput (left axis) and CPU utilization (right axis) as a function of the number of inference servers activated, each server executing on a vGPU within a 1g.5gb(7x) MIG configuration. requirements of multiple inference servers deployed within MIG. In the right axis …
Figure 10
Figure 10. Figure 10: High-level overview of PREBA. domain-specific acceleration. Concretely, all inference re￾quests routed to the MIG inference server are completely offloaded to our DPU, which is integrated at the PCIe bus as the host CPU’s co-processor, providing substantial improve￾me…
Figure 11
Figure 11. Figure 11: DPU microarchitecture for preprocessing (a) computer vision and (b) audio processing. Each functional unit within the CU handles its own preprocessing operation defined in [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 12
Figure 12. Figure 12: Execution timeline how the two requests (𝑋 and 𝑋+1) are handled by our DPU in (a) computer vision and (b,c) audio processing workloads. The timeline in (b) assumes a DPU design that integrates all of the functional units of audio processing within a single CU, whereas…
Figure 13
Figure 13. Figure 13: Histogram of Librispeech’s audio input lengths [67] GPUs are integrated within the same PCIe root complex, it is possible that our PREBA system can be bottlenecked by PCIe bandwidth constraints. In those circumstances, imple￾menting P2P data movement between DPU↔GPU a…
Figure 15
Figure 15. Figure 15: Tail latency of the CitriNet and two Conformer models over 1g.5gb(7x) when the audio input length changes among 5/15/25 seconds. Note that the tail latency value at 𝐵𝑎𝑡𝑐ℎ𝑘𝑛𝑒𝑒 (denoted as diamond markers) is almost constant around 35 ms (i.e., 𝑇𝑖𝑚𝑒𝑘𝑛𝑒𝑒 ), regardless of…
Figure 16
Figure 16. Figure 16: illustrates PREBA’s dynamic batching system that considers the variable-length nature of audio inputs and the corresponding 𝐵𝑎𝑡𝑐ℎ𝑘𝑛𝑒𝑒 and 𝐵𝑎𝑡𝑐ℎ𝑚𝑎𝑥 values for opti￾mal batching. In our proposed batching system, the audio input lengths are bucketized into multiple non-o…
Figure 17
Figure 17. Figure 17: Inference throughput of 1g.5gb(7x) when the number of activated inference servers is changed from one (1×) to seven (7×). Ideal Preprocessing (DPU) Preprocessing (CPU) 0 10 20 30 40 0 5000 10000 15000 Throughput (requests/sec) 0 10 20 30 40 0 10000 20000 30000 95%-ile…
Figure 18
Figure 18. Figure 18: Throughput vs. tail latency curve. All three designs run on top of a 1g.5gb(7x) MIG configuration. Model execution Preprocessing Batching Memcpy Others 0 3 6 9 12 CPU DPU Avg. latency (ms) 0 40 80 120 160 CPU DPU SqueezeNet Conformer (default) [PITH_FULL_IMAGE:figure…
Figure 19
Figure 19. Figure 19: Breakdown of end-to-end latency while running the experiments shown in [PITH_FULL_IMAGE:figures/full_fig_p011_19.png]
Figure 22
Figure 22. Figure 22: Ablation study showing the effectiveness of PREBA’s hardware and software proposals. Because the dynamic batch￾ing system targets audio processing algorithms, we only present speedup numbers for our audio processing AI workloads. Also, time is the active duration of t…
Figure 20
Figure 20. Figure 20: Power consumption (left) and energy-efficiency (right). 0 20 40 60 80 Baseline PREBA Baseline PREBA Baseline PREBA MobileNet SqueezeNet Swin￾Tansformer Cost efficiency (Million Req/$) (a) Computer vision 0 10 20 30 Baseline PREBA Baseline PREBA Baseline PREBA Critinet…
Figure 21
Figure 21. Figure 21: PREBA’s cost-efficiency (TCO). 6.2 Power and Energy-Efficiency We now evaluate PREBA’s effect on power consumption and energy-efficiency (Perf/Watt). In [PITH_FULL_IMAGE:figures/full_fig_p012_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 77 canonical work pages

  1. [1]

    https://www.graphcore.ai/products/c600

    GRAPHCORE. https://www.graphcore.ai/products/c600

  2. [2]

    https://github.com/ewan-xu/LibrosaCpp

    LibrosaCpp. https://github.com/ewan-xu/LibrosaCpp

  3. [3]

    https://opencv.org/

    OpenCV. https://opencv.org/

  4. [4]

    https://rebellions.ai/rebellions-product/atom-2/

    rebellions. https://rebellions.ai/rebellions-product/atom-2/

  5. [5]

    J. Ahn, D. Kwon, Y. Kim, M. Ajdari, J. Lee, and J. Kim. DCS: A Fast and Scalable Device-Centric Server Architecture. In Proceedings of the International Symposium on Microarchitecture (MICRO) , December 2015

    2015

  6. [6]

    Albericio, P

    J. Albericio, P. Judd, T. Hetherington, T. Aamodt, N. E. Jerger, and A. Moshovos. Cnvlutin: Ineffectual-Neuron-Free Deep Convolutional Neural Network Computing. In Proceedings of the International Sym- posium on Computer Architecture (ISCA) , June 2016

    2016

  7. [7]

    https://www.amazon.com/NVIDIA-Tesla-A100-Ampere- Graphics/dp/B0BGZJ27SL, 2024

    AMAZON. https://www.amazon.com/NVIDIA-Tesla-A100-Ampere- Graphics/dp/B0BGZJ27SL, 2024

    2024

  8. [8]

    Alveo U55C High Performance Compute Card

    AMD Xilinx. Alveo U55C High Performance Compute Card. https: //www.xilinx.com/products/boards-and-kits/alveo/u55c.html, 2023

    2023

Show all 94 references
  1. [9]

    AMD Vitis

    AMD Xilinx. AMD Vitis. https://www.xilinx.com/products/design- tools/vitis/vitis-hls.html, 2023

    2023

  2. [10]

    Vitis Libraries

    AMD Xilinx. Vitis Libraries. https://docs.amd.com/r/en-US/Vitis_ Libraries, 2024

    2024

  3. [11]

    Stanley Williams, Paolo Faraboschi, Wen mei Hwu, John Paul Strachan, Kaushik Roy, and Dejan S Milojicic

    Aayush Ankit, Izzat El Hajj, Sai Rahul Chalamalasetti, Geoffrey Ndu, Martin Foltin, R. Stanley Williams, Paolo Faraboschi, Wen mei Hwu, John Paul Strachan, Kaushik Roy, and Dejan S Milojicic. PUMA: A Pro- grammable Ultra-efficient Memristor-based Accelerator for Machine Learni...

    2019

  4. [12]

    Enabling Programmable Transport Protocols in High-Speed NICs

    Mina Tahmasbi Arashloo, Alexey Lavrov, Manya Ghobadi, Jennifer Rexford, David Walker, and David Wentzlaff. Enabling Programmable Transport Protocols in High-Speed NICs. In USENIX Symposium on Networked Systems Design and Implementation (NSDI) , 2020

    2020

  5. [13]

    AWS Nitro System

    AWS. AWS Nitro System. https://aws.amazon.com/ec2/nitro/, 2023

    2023

  6. [14]

    AWS inferentia

    A.W.Services. AWS inferentia. https://aws.amazon.com/machine- learning/inferentia/, 2024

    2024

  7. [15]

    Microsoft Announces Acquisition of Fungible to Ac- celerate Datacenter Innovation

    Girish Bablani. Microsoft Announces Acquisition of Fungible to Ac- celerate Datacenter Innovation. https://www.fungible.com, 2023

    2023

  8. [16]

    F4T: A Fast and Flexible FPGA-based Full- stack TCP Acceleration Framework

    Junehyuk Boo, Yujin Chung, Eunjin Baek, Seongmin Na, Changsu Kim, and Jangwoo Kim. F4T: A Fast and Flexible FPGA-based Full- stack TCP Acceleration Framework. In Proceedings of the International Symposium on Computer Architecture (ISCA) , 2023

    2023

  9. [17]

    Sheaffer, Sang-Ha Lee, and Kevin Skadron

    Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W. Sheaffer, Sang-Ha Lee, and Kevin Skadron. Rodinia: A Benchmark Suite for Heterogeneous Computing. InProceedings of the International Symposium on Workload Characterization (IISWC), 2009

    2009

  10. [18]

    Prophet: Precise QoS Prediction on Non- Preemptive Accelerators to Improve Utilization in Warehouse-Scale Computers

    Quan Chen, Hailong Yang, Minyi Guo, Ram Srivatsa Kannan, Jason Mars, and Lingjia Tang. Prophet: Precise QoS Prediction on Non- Preemptive Accelerators to Improve Utilization in Warehouse-Scale Computers. In Proceedings of the International Conference on Archi- tectural Support...

    2017

  11. [19]

    Baymax: QoS Awareness and Increased Utilization for Non-Preemptive Accelerators in Warehouse Scale Computers

    Quan Chen, Hailong Yang, Jason Mars, and Lingjia Tang. Baymax: QoS Awareness and Increased Utilization for Non-Preemptive Accelerators in Warehouse Scale Computers. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operation ...

    2016

  12. [20]

    Y. Chen, T. Krishna, J. Emer, and V. Sze. Eyeriss: An Energy-Efficient Re- configurable Accelerator for Deep Convolutional Neural Networks. In Proceedings of the International Solid State Circuits Conference (ISSCC) , February 2016

    2016

  13. [21]

    BM-Store: A Transparent and High-performance Lo- cal Storage Architecture for Bare-metal Clouds Enabling Large-scale Deployment

    Yiquan Chen, Jiexiong Xu, Chengkun Wei, Yijing Wang, Xin Yuan, Yangming Zhang, Xulin Yu, Yi Chen, Zeke Wang, Shuibing He, and Wenzhi Chen. BM-Store: A Transparent and High-performance Lo- cal Storage Architecture for Bare-metal Clouds Enabling Large-scale Deployment. In Procee...

    2023

  14. [22]

    Dlbooster: Boosting End-to- End Deep Learning Workflows with Offloading Data Preprocessing Pipelines

    Yang Cheng, Dan Li, Zhiyuan Guo, Binyao Jiang, Jiaxin Lin, Xi Fan, Jinkun Geng, Xinyi Yu, Wei Bai, Lei Qu, Ran Shu, Peng Cheng, Yongqiang Xiong, and Jianping Wu. Dlbooster: Boosting End-to- End Deep Learning Workflows with Offloading Data Preprocessing Pipelines. In Proceeding...

    2019

  15. [23]

    Lazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference

    Yujeong Choi, Yunseong Kim, and Minsoo Rhu. Lazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference. In Proceedings of the International Symposium on High-Performance Computer Architecture (HPCA), 2021

    2021

  16. [24]

    PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Units

    Yujeong Choi and Minsoo Rhu. PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Units. In Proceedings of the International Symposium on High-Performance Com- puter Architecture (HPCA), 2020

    2020

  17. [25]

    Clipper: A Low-Latency Online Prediction Serving System

    Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J Franklin, Joseph E Gonzalez, and Ion Stoica. Clipper: A Low-Latency Online Prediction Serving System. In Proceedings of USENIX Symposium on Networked Systems Design and Implementation (NSDI) , 2017

    2017

  18. [26]

    Everything you need to know about data center power

    DATACENTERS.com. Everything you need to know about data center power. https://www.datacenters.com/news/everything-you-need-to- know-about-data-center-power , 2020

    2020

  19. [27]

    Neural Cache: Bit-serial In-cache Acceleration of Deep Neural Networks

    Charles Eckert, Xiaowei Wang, Jingcheng Wang, Arun Subramaniyan, Ravi Iyer, Dennis Sylvester, David Blaauw, and Reetuparna Das. Neural Cache: Bit-serial In-cache Acceleration of Deep Neural Networks. In Proceedings of the International Symposium on Computer Architecture (ISCA), 2018

    2018

  20. [28]

    MAICC : A Lightweight Many-core Architecture with In-Cache Computing for Multi-DNN Parallel Infer- ence

    Renhao Fan, Yikai Cui, Qilin Chen, Mingyu Wang, Youhui Zhang, Weimin Zheng, and Zhaolin Li. MAICC : A Lightweight Many-core Architecture with In-Cache Computing for Multi-DNN Parallel Infer- ence. In Proceedings of the International Symposium on Microarchitec- ture (MICRO), 2023

    2023

  21. [29]

    Low Latency RNN Inference with Cellular Batching

    Pin Gao, Lingfan Yu, Yongwei Wu, and Jinyang Li. Low Latency RNN Inference with Cellular Batching. In Proceedings of the EuroSys Conference, 2018

    2018

  22. [30]

    Low Latency RNN Inference with Cellular Batching

    Pin Gao, Lingfan Yu, Yongwei Wu, and Jinyang Li. Low Latency RNN Inference with Cellular Batching. In Proceedings of the EuroSys Conference (EuroSys), 2018

    2018

  23. [31]

    Cachew: Machine Learning Input Data Processing as a Service

    Dan Graur, Damien Aymon, Dan Kluser, Tanguy Albrici, Chandramo- han A Thekkath, and Ana Klimovic. Cachew: Machine Learning Input Data Processing as a Service. In USENIX Annual Technical Conference (USENIX ATC), 2022

    2022

  24. [32]

    Conformer: Convolution-augmented Transformer for Speech Recognition

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, and Yonghui Wu. Conformer: Convolution-augmented Transformer for Speech Recognition. 2020

    2020

  25. [33]

    S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. Horowitz, and W. Dally. EIE: Efficient Inference Engine on Compressed Deep Neural Network. In Proceedings of the International Symposium on Computer Architecture (ISCA), June 2016

    2016

  26. [34]

    Hauswald, Y

    J. Hauswald, Y. Kang, M. A. Laurenzano, Q. Chen, C. Li, T. Mudge, R. G. Dreslinski, J. Mars, and L. Tang. DjiNN and Tonic: DNN as a Service and Its Implications for Future Warehouse Scale Computers. In Proceedings of the International Symposium on Computer Architecture (ISCA),...

    2015

  27. [35]

    Le, and Hartwig Adam

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, and Hartwig Adam. Searching for Mo- bileNetV3. arXiv preprint arXiv:1905.02244, 2019

    1905 arXiv

  28. [36]

    SK hynix Develops World’s Best Performing HBM3E, Provides Samples to Customer for Performance Evaluation

    Hynix. SK hynix Develops World’s Best Performing HBM3E, Provides Samples to Customer for Performance Evaluation. 2023

    2023

  29. [37]

    SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and <0.5MB Model Size

    Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and <0.5MB Model Size. arXiv preprint arXiv:1602.07360, 2016. 13

    2016 arXiv

  30. [38]

    ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012)

    ImageNet. ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012). https://www.image-net.org/challenges/LSVRC/2012/, 2012

    2012

  31. [39]

    Intel Infrastructure Processing Unit (Intel IPU).https://www.intel

    Intel. Intel Infrastructure Processing Unit (Intel IPU).https://www.intel. com/content/www/us/en/products/details/network-io/ipu.html, 2023

    2023

  32. [40]

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers, R. Boyle, P. Cantin, C. Chao, C. Clark, J. Coriell, M. Daley, M. Dau, J. Dean, B. Gelb, T. V. Ghaem- maghami, R. Gottipati, W. Gulland, R. Hagmann, C. R. Ho, D. Ho...

    2017

  33. [41]

    Khronos Group. OpenCL. https://www.khronos.org/opencl/, 2023

    2023

  34. [42]

    Rearchitecting the TCP Stack for I/O- Offloaded Content Delivery

    Taehyun Kim, Deondre Martin Ng, Junzhi Gong, Youngjin Kwon, Min- lan Yu, and KyoungSoo Park. Rearchitecting the TCP Stack for I/O- Offloaded Content Delivery. In USENIX Symposium on Networked Systems Design and Implementation (NSDI) , 2023

    2023

  35. [43]

    PARIS and ELSA: An Elastic Scheduling Algorithm for Reconfigurable Multi-GPU Inference Servers

    Yunseong Kim, Yujeong Choi, and Minsoo Rhu. PARIS and ELSA: An Elastic Scheduling Algorithm for Reconfigurable Multi-GPU Inference Servers. In Proceedings of the ACM/IEEE Design Automation Conference (DAC), 2022

    2022

  36. [44]

    Dcs-ctrl: A Fast and Flexible Device-Control Mechanism for Device-Centric Server Architecture

    Dongup Kwon, Jaehyung Ahn, Dongju Chae, Mohammadamin Ajdari, Jaewon Lee, Suheon Bae, Youngsok Kim, and Jangwoo Kim. Dcs-ctrl: A Fast and Flexible Device-Control Mechanism for Device-Centric Server Architecture. In Proceedings of the International Symposium on Computer Architec...

    2018

  37. [45]

    FVM: FPGA-assisted Virtual Device Emulation for Fast, Scalable, and Flexible Storage Virtualization

    Dongup Kwon, Junehyuk Boo, Dongryeong Kim, and Jangwoo Kim. FVM: FPGA-assisted Virtual Device Emulation for Fast, Scalable, and Flexible Storage Virtualization. In USENIX Symposium on Operating Systems Design and Implementation (OSDI) , 2020

    2020

  38. [46]

    A Fast and Flexible Hardware-based Virtualization Mechanism for Computational Storage Devices

    Dongup Kwon, Dongryeong Kim, Junehyuk Boo, Wonsik Lee, and Jangwoo Kim. A Fast and Flexible Hardware-based Virtualization Mechanism for Computational Storage Devices. In USENIX Annual Technical Conference (USENIX ATC), 2021

    2021

  39. [47]

    Char- acterizing Multi-Instance GPU for Machine Learning Workloads

    Baolin Li, Viiay Gadepally, Siddharth Samsi, and Devesh Tiwari. Char- acterizing Multi-Instance GPU for Machine Learning Workloads. In Proceedings of IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2022

    2022

  40. [48]

    MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant GPU Clusters

    Baolin Li, Tirthak Patel, Siddharth Samsi, Vijay Gadepally, and De- vesh Tiwari. MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant GPU Clusters. In Proceedings of the Symposium on Cloud Computing (SoCC), 2022

    2022

  41. [49]

    Leapio: Efficient and Portable Virtual NVMe Storage on ARM SOCs

    Huaicheng Li, Mingzhe Hao, Stanko Novakovic, Vaibhav Gogte, Sriram Govindan, Dan RK Ports, Irene Zhang, Ricardo Bianchini, Haryadi S Gunawi, and Anirudh Badam. Leapio: Efficient and Portable Virtual NVMe Storage on ARM SOCs. In Proceedings of the International Conference on Ar...

    2020

  42. [50]

    E3: Energy-efficient Microservices on SmartNIC- accelerated Servers

    Ming Liu, Simon Peter, Arvind Krishnamurthy, and Phitchaya Mangpo Phothilimthana. E3: Energy-efficient Microservices on SmartNIC- accelerated Servers. In USENIX Annual Technical Conference (USENIX ATC), 2019

    2019

  43. [51]

    Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. InProceedings of the International Conference on Computer Vision (ICCV) , 2021

    2021

  44. [52]

    Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition, 2021

    Somshubra Majumdar, Jagadeesh Balam, Oleksii Hrinchuk, Vitaly Lavrukhin, Vahid Noroozi, and Boris Ginsburg. Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition, 2021

    2021

  45. [53]

    Boost Your Datacenter! https://mangoboost.io/, 2023

    MangoBoost. Boost Your Datacenter! https://mangoboost.io/, 2023

    2023

  46. [54]

    Meta MITA v1

    Meta. Meta MITA v1. https://ai.meta.com/blog/meta-training- inference-accelerator-AI-MTIA/ , 2023

    2023

  47. [55]

    Gimbal: Enabling Multi- Tenant Storage Disaggregation on SmartNIC JBOFs

    Jaehong Min, Ming Liu, Tapan Chugh, Chenxingyu Zhao, Andrew Wei, In Hwan Doh, and Arvind Krishnamurthy. Gimbal: Enabling Multi- Tenant Storage Disaggregation on SmartNIC JBOFs. In Proceedings of the ACM SIGCOMM Conference, 2021

    2021

  48. [56]

    Analyzing and Mitigating Data Stalls in DNN Training

    Jayashree Mohan, Amar Phanishayee, Ashish Raniwala, and Vijay Chidambaram. Analyzing and Mitigating Data Stalls in DNN Training. Proceedings of the VLDB Endowment , 2021

    2021

  49. [57]

    AccelTCP: Accelerating Network Applications with Stateful TCP Offloading

    YoungGyoun Moon, SeungEon Lee, Muhammad Asim Jamshed, and KyoungSoo Park. AccelTCP: Accelerating Network Applications with Stateful TCP Offloading. In USENIX Symposium on Networked Systems Design and Implementation (NSDI) , 2020

    2020

  50. [58]

    NVIDIA Triton Inference Server

    NVIDIA. NVIDIA Triton Inference Server. https://developer.nvidia. com/nvidia-triton-inference-server

  51. [59]

    NVIDIA TensorRT: Programmable Inference Accelerator

    NVIDIA. NVIDIA TensorRT: Programmable Inference Accelerator. 2018

    2018

  52. [60]

    NVIDIA T4

    NVIDIA. NVIDIA T4. https://www.nvidia.com/content/dam/en- zz/Solutions/Data-Center/tesla-t4/t4-tensor-core-datasheet- 951643.pdf, 2019

    2019

  53. [61]

    NVIDIA A100

    NVIDIA. NVIDIA A100. https://www.nvidia.com/content/dam/en- zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet.pdf , 2020

    2020

  54. [62]

    NVIDIA CUDA Programming Guide, 2021

    NVIDIA. NVIDIA CUDA Programming Guide, 2021

    2021

  55. [63]

    Multi-Instance GPU

    NVIDIA. Multi-Instance GPU. https://www.nvidia.com/en-us/ technologies/multi-instance-gpu/ , 2023

    2023

  56. [64]

    NVIDIA BlueField Data Processing Units

    NVIDIA. NVIDIA BlueField Data Processing Units. https://www. nvidia.com/en-us/networking/products/data-processing-unit/ , 2023

    2023

  57. [65]

    NVIDIA NeMo

    NVIDIA. NVIDIA NeMo. https://www.nvidia.com/en-us/ai-data- science/generative-ai/nemo-framework/, 2023

    2023

  58. [66]

    TensorFlow-Serving: Flexible, High-Performance ML Serving

    Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fangwei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. TensorFlow-Serving: Flexible, High-Performance ML Serving. arXiv preprint arXiv:1712.06139, 2017

    2017 arXiv

  59. [67]

    LibriSpeech ASR corpus

    Open SLR. LibriSpeech ASR corpus. http://www.openslr.org/12/

  60. [68]

    Parashar, M

    A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally. SCNN: An Ac- celerator for Compressed-sparse Convolutional Neural Networks. In Proceedings of the International Symposium on Computer Architecture (ISCA), June 2017

    2017

  61. [69]

    TrainBox: An Extreme-Scale Neural Network Training Server Architecture by Sys- tematically Balancing Operations

    Pyeongsu Park, Heetaek Jeong, and Jangwoo Kim. TrainBox: An Extreme-Scale Neural Network Training Server Architecture by Sys- tematically Balancing Operations. In Proceedings of the International Symposium on Microarchitecture (MICRO) , 2020

    2020

  62. [70]

    Single-Root Input/Output Virtualization

    PCI SIG. Single-Root Input/Output Virtualization. https://www.pcisig. com/specifications

  63. [71]

    Torchaudio Transforms

    PyTorch. Torchaudio Transforms. https://pytorch.org/audio/stable/ transforms.html

  64. [72]

    Transforming and Augmenting Images

    PyTorch. Transforming and Augmenting Images. https://pytorch.org/ vision/stable/transforms.html

  65. [73]

    PyTorch Hub

    PyTorch. PyTorch Hub. https://pytorch.org/hub/, 2023

    2023

  66. [74]

    Scott Gardner, Itay Hubara, Sachin Idgunji, Thomas B

    Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximi- lien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman, Sam Davis, Pan Deng, Greg Diamos, Jared Duke, Dave Fick, J. Scott Gardner, ...

    2020

  67. [75]

    An Analysis of Collocation on GPUs for Deep Learning Training

    Ties Robroek, Ehsan Yousefzadeh-Asl-Miandoab, and Pınar Tözün. An Analysis of Collocation on GPUs for Deep Learning Training. arXiv preprint arXiv:2209.06018, 2022

    2022 arXiv

  68. [76]

    Samsung Develops Industry’s First GDDR7 DRAM To Un- lock the Next Generation of Graphics Performance

    Samsung. Samsung Develops Industry’s First GDDR7 DRAM To Un- lock the Next Generation of Graphics Performance. 2023

    2023

  69. [77]

    Tell, Yanqing Zhang, William J

    Yakun Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matthew Fojtik, Nan Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Stephen G. Tell, Yanqing Zhang, William J. Dally, Joel Emer, C. Thomas Gray, Brucek Khailany, and Stephen W. K...

    2019

  70. [78]

    Laconic Deep Learning Inference Acceleration

    Sayeh Sharify, Alberto Delmas Lascorz, Mostafa Mahmoud, Milos Nikolic, Kevin Siu, Dylan Malone Stuart, Zissis Poulos, and Andreas Moshovos. Laconic Deep Learning Inference Acceleration. In Proceed- ings of the International Symposium on Computer Architecture (ISCA) , 2019

    2019

  71. [79]

    Bit Fu- sion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network

    Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, and Hadi Esmaeilzadeh. Bit Fu- sion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network. In Proceedings of the International Symposium on Compute...

    2018

  72. [80]

    FlexTOE: Flexible TCP Offload with Fine-Grained Parallelism

    Rajath Shashidhara, Tim Stamler, Antoine Kaufmann, and Simon Peter. FlexTOE: Flexible TCP Offload with Fine-Grained Parallelism. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2022

    2022

  73. [81]

    Microsoft Maia v1

    Jake Siegel. Microsoft Maia v1. https://news.microsoft.com/source/ features/ai/in-house-chips-silicon-to-service-to-meet-ai-demand/ , 2023

    2023

  74. [82]

    https://store.supermicro.com/us_en/mainstream-amd- 2u-as-2024s-tr.html , 2024

    SuperMICRO. https://store.supermicro.com/us_en/mainstream-amd- 2u-as-2024s-tr.html , 2024

    2024

  75. [83]

    Lynx: A SmartNIC- Driven Accelerator-Centric Architecture for Network Servers

    Maroun Tork, Lina Maudlej, and Mark Silberstein. Lynx: A SmartNIC- Driven Accelerator-Centric Architecture for Network Servers. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operation Systems (ASPLOS) , 2020

    2020

  76. [84]

    FastFlow: Accelerating Deep Learn- ing Model Training with Smart Offloading of Input Data Pipeline

    Taegeon Um, Byungsoo Oh, Byeongchan Seo, Minhyeok Kweun, Goeun Kim, and Woo-Yeon Lee. FastFlow: Accelerating Deep Learn- ing Model Training with Smart Offloading of Input Data Pipeline. Proceedings of the VLDB Endowment , 2023

    2023

  77. [85]

    Real-time Meets Approximate Computing: An Elastic CNN Inference Accelerator with Adaptive Trade-off Between QoS and QoR

    Ying Wang, Huawei Li, and Xiaowei Li. Real-time Meets Approximate Computing: An Elastic CNN Inference Accelerator with Adaptive Trade-off Between QoS and QoR. In Design Automation Conference (DAC), 2017

    2017

  78. [86]

    A None- Sparse Inference Accelerator that Distills and Reuses the Computation Redundancy in CNNs

    Ying Wang, Shengwen Liang, Huawei Li, and Xiaowei Li. A None- Sparse Inference Accelerator that Distills and Reuses the Computation Redundancy in CNNs. In Design Automation Conference (DAC), 2019

    2019

  79. [87]

    FpgaNIC: An FPGA-based Versatile 100Gb SmartNIC for GPUs

    Zeke Wang, Hongjing Huang, Jie Zhang, Fei Wu, and Gustavo Alonso. FpgaNIC: An FPGA-based Versatile 100Gb SmartNIC for GPUs. In USENIX Annual Technical Conference (USENIX ATC), 2022

    2022

  80. [88]

    Xilinx OpenCL Extension

    Xilinx. Xilinx OpenCL Extension. https://xilinx.github.io/XRT/master/ html/opencl_extension.html, 2022

    2022

  81. [89]

    Vitis High-level Synthesis User Guide

    Xilinx. Vitis High-level Synthesis User Guide. https://docs.xilinx.com/ r/en-US/ug1399-vitis-hls , 2023

    2023

  82. [90]

    https://www.xilinx.com/products/boards-and-kits/alveo/u55c

    Xilinx. https://www.xilinx.com/products/boards-and-kits/alveo/u55c. html, 2024

    2024

  83. [91]

    Sinclair, Bradford M

    Tsung Tai Yeh, Matthew D. Sinclair, Bradford M. Beckmann, and Tim- othy G. Rogers. Deadline-Aware Offloading for High-Throughput Accelerators. In Proceedings of the International Symposium on High- Performance Computer Architecture (HPCA), 2021

    2021

  84. [92]

    Orca: A Distributed Serving System for Transformer-Based Generative Models

    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. Orca: A Distributed Serving System for Transformer-Based Generative Models. In USENIX Symposium on Operating Systems Design and Implementation (OSDI) , 2022

    2022

  85. [93]

    MArk: Ex- ploiting Cloud Services for Cost-Effective,SLO-Aware Machine Learn- ing Inference Serving

    Chengliang Zhang, Minchen Yu, Wei Wang, and Feng Yan. MArk: Ex- ploiting Cloud Services for Cost-Effective,SLO-Aware Machine Learn- ing Inference Serving. In Proceedings of USENIX Annual Technical Conference (ATC), 2019

    2019

  86. [94]

    Understand- ing Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training: Industrial Product

    Mark Zhao, Niket Agarwal, Aarti Basant, Buğra Gedik, Satadru Pan, Mustafa Ozdal, Rakesh Komuravelli, Jerry Pan, Tianshu Bao, Haowei Lu, Sundaram Narayanan, Jack Langman, Kevin Wilfong, Harsha Ras- togi, Carole-Jean Wu, Christos Kozyrakis, and Parik Pol. Understand- ing Data St...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.