{"id":"d0d2fb72-9fae-4c35-b0e1-8b8283808d48","arxiv_id":"2507.08954","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MQFQ-Sticky applies multi-queue fair queuing and anticipatory scheduling to GPU serverless functions, reporting 2x-20x latency reductions and fairer GPU service than FCFS, batching, or SJF policies.","lead":"MQFQ-Sticky is a serverless GPU scheduler that borrows I/O fair-queueing ideas to reduce latency for containerized GPU functions. It combines per-function queues, warm container pools, and GPU memory prefetch/swap, reporting 2x to 20x lower latency than common queueing policies.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'black-box' and 2x-20x latency claims depend on an LD_PRELOAD shim that only rewrites cuMemAlloc; functions using cuMemMap/cuMemCreate, CUDA graphs, or static CUDA links bypass it, so the claimed generality is unverified for exactly the high-performance functions the paper targets.","rationale":"The reader identified the right load-bearing assumption: the paper's core promise is black-box generality, and the reported latency reductions depend on proactive memory management, which in turn depends on intercepting all device allocations. The shim mechanism in Section 5.1 only rewrites cuMemAlloc, so functions that use newer CUDA allocation paths, CUDA graphs, custom allocators, or static CUDA linkage fall outside the mechanism. For those functions, the warm-pool, prefetch, and swap optimizations either do nothing or cannot safely manage memory, and the claimed 2x-20x latency improvement is not supported. This is a coverage gap internal to the stated mechanism, not a disagreement with external consensus, and it directly affects the central claim. I also note the secondary Algorithm 1 state-assignment issue: line 22 marks a queue Throttled exactly when the line 6 dispatch filter would make it eligible, which is internally inconsistent with the fairness bound. Since the implementation is not released, this is a reproducibility concern rather than the main conceptual weakness. The empirical head-to-head comparisons are the right kind of evidence, and the paper gives credit for using the same memory-management substrate across all queueing policies. Given the missing artifact and the shim coverage gap, CONDITIONAL remains the correct verdict.","tokens_in":23922,"tokens_out":9764,"duration_ms":130824,"concrete_test":"Run a representative CUDA function that allocates device memory via cuMemCreate/cuMemMap under MQFQ-Sticky's shim, first with a working set within GPU memory and then with allocations exceeding device memory; additionally run a CUDA-graph-based function using a memory pool. If either function produces wrong results, crashes, or exhibits more than 10% slowdown relative to native execution, the black-box generality claim fails and the conclusions must be scoped to cuMemAlloc-based functions. Also run a statically linked CUDA binary to confirm that the LD_PRELOAD path is bypassed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central promise is black-box generality: Section 3.1 says functions are run without modifying the code or making assumptions about it, and the latency gains rely on the shim-based memory management in Section 4.3 that tracks, prefetches, and swaps allocations. Section 5.1 shows the shim intercepts only cuMemAlloc and rewrites it to cuMemAllocManaged. This leaves a meaningful class of CUDA functions uncovered: those using cuMemCreate/cuMemMap (the VMM API), CUDA graphs with memory pools, custom allocators that avoid cuMemAlloc, or statically linked CUDA runtimes. For those functions, the scheduler cannot know or control their memory footprint; if the working set exceeds physical VRAM, UVM oversubscription is not applied and the function either fails or thrashes. The evaluation covers 14 functions, and even Srad shows a 30% execution-time overhead in Figure 3, so negligible overhead and complete semantic preservation are already not universally true. The paper does not state this scope limitation, so the abstract's black-box and 2x-20x latency claims overgeneralize from cuMemAlloc-based workloads.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MQFQ-Sticky, a GPU function scheduler for serverless FaaS that adapts Multi-Queue Fair Queuing and anticipatory scheduling to the GPU setting, and integrates it with UVM-based memory prefetching and swapping inside a container pool. The system targets black-box containerized CUDA functions, claims to balance locality, fairness, and latency, and reports latency reductions of 2x--20x over existing queueing policies and over 300x over unoptimized GPU containers. The evaluation uses two GPU platforms, a mix of machine-learning, video, and HPC functions, open-loop Zipfian and Azure-derived workloads, and a sensitivity study of the main parameters. The central design idea is plausible and the empirical scope is broad, but several issues in the pseudocode, the fairness argument, and the claimed black-box generality need to be addressed.","tokens_in":24142,"tokens_out":5481,"duration_ms":67893,"significance":"If the central claims hold, this is a useful contribution to serverless GPU systems: it provides a principled I/O-scheduling-inspired approach to GPU function scheduling, integrates memory management with queue state, and demonstrates robustness across parameter choices and hardware features. The work is distinctive in aiming for black-box function support rather than application-specific optimizations, and it explicitly measures fairness, locality, and latency tradeoffs. The empirical evaluation is broad, including two GPU generations, MPS/MIG, multi-GPU settings, and multiple workload classes. However, the significance is tempered by the fact that the fairness bound is inherited from prior work rather than derived, the black-box claim is only validated for cuMemAlloc-based functions, and the experimental reporting lacks confidence intervals and an artifact release.","major_comments":[{"comment":"The black-box generality claim is broader than what the implementation supports. Section 5.1 states that the LD_PRELOAD shim intercepts only cuMemAlloc and rewrites it to cuMemAllocManaged, and records metadata for memory management. Functions that use the CUDA VMM API (cuMemCreate/cuMemMap), CUDA graphs with memory pools, custom allocators layered above cuMemAlloc, or statically linked CUDA runtimes will bypass the shim entirely; for those functions, the scheduler cannot track or prefetch/swap GPU memory, so the UVM-based oversubscription and proactive memory management do not apply. The paper's abstract and Section 3.1 claim black-box operation without modifying function code, and the latency and cold-start improvements in the evaluation depend directly on the memory management in Section 4.3. This is a load-bearing limitation: the 2x--20x latency claim is demonstrated only for functions whose allocations go through cuMemAlloc. The paper should either state this scope explicitly in the abstract and Section 3.1, or extend the shim to cover the VMM API and CUDA graph memory pools, and discuss the expected impact on the reported results. The 30% overhead reported for Srad in Figure 3 also shows that the 'negligible overhead' characterization is not universally true even within the supported class.","section":"Section 5.1 / Section 3.1"},{"comment":"There is a direct inconsistency between the candidate-selection predicate and the state-update rule. Line 6 selects active queues with queue.VT < Global_VT + T, which is the eligibility condition described in the text. Line 22, however, sets a queue to Throttled when queue.VT - Global_VT < T, which is exactly the same condition under a sign-inverted formulation; as written, the queues that line 6 would select are marked throttled, while queues with queue.VT - Global_VT >= T are marked active even though line 6 would not dispatch them. This appears to be a sign error in the pseudocode, but since Algorithm 1 is the formal specification of the proposed scheduler, it must be corrected and the corrected version should be checked against the prose in Section 4.2, which itself contains the confusing statement 'If queue.VT + T >= Global_VT, then the queue is throttled'.","section":"Algorithm 1"},{"comment":"The fairness bound is claimed to follow from the MQFQ theorem [40], but the scheduler advances virtual time using the historical average execution time tau_k rather than the actual service time of the dispatch. Section 4.2 says 'we track the historical average execution time tau_k ... and when an item is dispatched, increment its queue's VT by tau_k.' Equation (1), however, bounds the difference in actual GPU service times S_i and S_j. If the actual execution time of an invocation deviates from tau_k, the virtual-time accounting no longer reflects service actually rendered, and the MQFQ fairness bound is not automatically inherited. The paper needs to state the assumptions under which Eq. (1) holds, or restate the bound in terms of the estimated virtual time; otherwise, the 'we retain the fairness properties of MQFQ' claim in Section 4.2 is not justified. The empirical check in Figure 5b is suggestive but does not substitute for a correct theoretical statement.","section":"Section 4.2"},{"comment":"The empirical central claims would be materially strengthened by reporting variability and providing the artifact. Most figures show averages over 5 runs without confidence intervals or per-run distributions, so it is difficult to assess whether the reported 2x--5x improvements over FCFS or the differences between MQFQ-Sticky and Batch are statistically meaningful for the selected traces. Additionally, no artifact or raw data is provided, and the Paella and Batch baselines are reimplementations rather than the original systems. Given that the paper claims a new scheduler with large latency gains and a fairness guarantee, the lack of an artifact makes independent verification hard and should be addressed, at minimum by releasing the scheduler, the shim, and the workload traces, and by including confidence intervals or run-level data in the main latency comparisons.","section":"Section 6"}],"minor_comments":[{"comment":"The text says that line 9 of Algorithm 1 implements 'sticky' load balancing among GPUs by avoiding moving functions across GPUs, but line 9 only sorts candidates by in-flight count; no GPU-affinity or sticky-binding mechanism is visible in Algorithm 1. Please clarify how the dispatcher maintains per-GPU locality.","section":"Section 5"},{"comment":"The captions of Figures 6 and 7 both read 'Latency, fairness, and utilization for a medium-intensity FaaS workload', but the figures mainly show latency and utilization; no fairness measure is presented in either figure. Please make the captions reflect the content.","section":"Figures 6 and 7"},{"comment":"The comparison to 'FCFS Naive' with an average latency near 3,000 seconds is described but not shown in any figure; since this is the basis for the '300x' claim, please add a table or a note describing how this number was obtained.","section":"Section 6.2"},{"comment":"There are typos in the related work section: 'stravation' should be 'starvation', and 'investigtes' should be 'investigates'.","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"The core scheduling idea is reasonable and the evaluation is broad, but the manuscript needs a careful pass over the pseudocode and the fairness argument before it can be accepted. The sign inconsistency in Algorithm 1 and the unstated reliance on cuMemAlloc-based functions are fixable but should be resolved. I would not recommend reject: the contribution is potentially useful and the empirical evidence is substantial, but the claims currently outrun the implementation and the proof. If the authors release the artifact and add confidence intervals, this would strengthen the paper substantially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MQFQ-Sticky is a solid systems contribution. It borrows multi-queue fair queuing and anticipatory scheduling from disk I/O, applies them to black-box GPU function dispatch, and pairs them with UVM-based prefetch/swap and a warm container pool. The result: 2x-20x latency reductions over FCFS and other policies, with much lower variance and demonstrable fairness across heterogeneous functions. That's real and worth taking seriously.\n\nWhat's genuinely new here is the integration. Prior GPU-serverless work focuses on inference-specific batching or kernel-level remoting; this is a general-purpose scheduler that treats each function as a queue and uses virtual time to balance locality and fairness. The sticky heuristic (prefer long queues, break ties by in-flight count) is simple and works. The evaluation is broad: two very different GPUs, multiple Azure-sampled and Zipfian workloads, MPS/MIG/multi-GPU, and a sensitivity study of the main parameters. I believe the empirical claims.\n\nThe soft spots are real but not fatal. First, the 'black-box' promise is narrower than the abstract suggests. The LD_PRELOAD shim intercepts cuMemAlloc only; functions that use cuMemMap/cuMemCreate, CUDA graphs, static CUDA links, or custom allocators bypass it. The paper doesn't state this limitation, and the 30% overhead on Srad shows 'transparent' isn't always true. That's a scope issue, not a design flaw. Second, there's no artifact or raw data despite the 'first open-source' claim. Third, most figures are 5-run averages without confidence intervals, which is below the bar for a top systems venue. Fourth, the throttling condition in Algorithm 1 (line 22) looks inverted relative to the text and the filter condition; the bound they cite from MQFQ is inherited, not re-derived. These are cleanly fixable in revision.\n\nOverall, this paper deserves a serious referee. The core idea is sound, the evaluation is convincing, and the missing pieces are identifiable. I'd send it to peer review with a request for the artifact and a fix for the pseudocode. The reader's conditional verdict is about right. For systems researchers working on serverless GPU scheduling, this is a useful reference.","headline":"MQFQ-Sticky is a solid systems contribution that adapts fair queueing to GPU function dispatch; the main caveat is that the 'black-box' claim is narrower than advertised because the shim only intercepts cuMemAlloc.","tokens_in":24738,"tokens_out":4460,"would_cite":true,"duration_ms":44757,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MQFQ-Sticky schedules GPU functions with I/O-style fair queueing and reports 2x-20x latency reductions.","keywords":["MQFQ-Sticky","serverless computing","GPU scheduling","fair queueing","anticipatory scheduling","Unified Virtual Memory","GPU memory management","warm container pool"],"falsifier":"Run a workload of CUDA functions that link the CUDA runtime statically or allocate memory through low-level interfaces the shim does not intercept, and check whether they run correctly and keep the reported 2x-20x gains; one such function failing or showing the kind of 30% slowdown measured for Srad would break the black-box claim.","tokens_in":23697,"feed_emoji":"⚡","tokens_out":6427,"duration_ms":62784,"temperature":0.7,"pith_summary":"This paper tries to show that GPU-accelerated serverless functions can be scheduled efficiently without modifying user code, by transplanting fair-queueing ideas from disk I/O scheduling into the FaaS control plane. It argues that GPU function dispatch has the same locality-versus-fairness tension as I/O scheduling, and that a locality-enhanced variant of multi-queue fair queuing, paired with proactive Unified Virtual Memory prefetch and swap, resolves it. The authors claim this reduces average function latency by 2x to 20x compared with existing GPU and CPU queueing policies, and by more than 300x compared with unoptimized GPU containers. If correct, serverless platforms could offer near-specialized-inference latency and utilization while still running arbitrary black-box functions.","feed_headline":"Fair-queueing scheduler cuts GPU function latency up to 20x","feed_subtitle":"Turning GPU dispatch into an I/O scheduling problem plus smart memory reuse gives fast, fair, black-box acceleration.","key_machinery":"The central object is MQFQ (Multi-Queue Fair Queuing) transplanted from I/O scheduling, adapted to GPU function dispatch. Each function gets a queue with a virtual time (VT) equal to accumulated GPU service time; a dispatcher picks among queues whose VT is within T of the global minimum VT, sorts by backlog length then in-flight count, and the device-parallelism parameter D controls how many invocations run concurrently. Two GPU-specific mechanisms carry the performance argument: anticipatory scheduling keeps empty queues alive for a TTL proportional to alpha times the function's inter-arrival time, and an integrated memory manager uses an LD_PRELOAD shim that rewrites cuMemAlloc to cuMemAllocManaged, prefetching active functions' memory to the GPU and asynchronously swapping throttled or inactive memory back to host via LRU. The fairness guarantee, inherited from MQFQ, bounds the difference in normalized GPU service time across any two backlogged functions.","core_discovery":"The paper's central claim is that MQFQ-Sticky, an integrated GPU function scheduler and memory manager, balances the tradeoffs between locality, fairness, and latency for black-box containerized functions. Viewing the GPU as a multi-queue I/O device, it keeps one dispatch queue per function, tracks virtual service time per queue, throttles queues that exceed a global minimum by a tunable overrun T, and dispatches candidate queues by longest queue then fewest in-flight invocations to create batching and stickiness. An anticipatory keep-alive TTL keeps empty queues alive in proportion to their inter-arrival time, and a warm container pool with CUDA UVM prefetch/swap moves function memory onto and off the device ahead of execution. The authors show empirically that service time across functions tracks the MQFQ fairness bound, and that the policy reduces latency versus FCFS, continuous batching, and Paella-style fair SJF on Zipfian and Azure-sampled workloads.","pith_inferences":["Beyond the paper, the GPU-as-multi-queue-I/O-device analogy suggests cluster-level load balancers could use per-server virtual-time state instead of consistent hashing to preserve locality across servers; MQFQ-Sticky currently treats load balancing as separate.","A testable extension is applying the same anticipatory TTL and queue-overrun parameters to other accelerators with scarce on-device memory and expensive context initialization, such as TPU or NPU slices with unified addressing; the paper only evaluates NVIDIA GPUs.","The measured 30% Srad shim overhead implies a production deployment would need a non-UVM fallback path for UVM-sensitive kernels, and the paper does not quantify how such fallbacks would affect the fairness bound.","Because MQFQ-Sticky's dispatch heuristic selects deterministically among queues that are MQFQ-legal, a tighter fairness bound than the inherited one may be derivable; the paper notes this possibility and leaves it open."],"forward_implications":["If MQFQ-Sticky works as claimed, GPU FaaS platforms can keep a warm container pool inside a few tens of GB of VRAM by swapping to host memory, cutting cold-start latency by orders of magnitude for black-box functions.","Fairness is preserved: the difference in GPU service time received by any two backlogged functions stays within the MQFQ bound, so popular functions cannot starve rare ones.","The scheduler composes with hardware multiplexing layers: with MPS it reduces latency by up to 80%, with MIG it exposes the slowdown small slices cause, and with multiple GPUs it extends locality-aware dispatch to cut queuing by up to 4x.","The parameter space of overrun T, parallelism D, keep-alive TTL, and pool size has a wide stable region, so operators can tune for locality versus fairness without brittle configurations.","All comparison policies in the evaluation use the same memory-management and shim optimizations, isolating queueing policy as the differentiator."],"supporting_citations":[{"why":"Supplies the start-time fair queueing virtual-time framework that MQFQ builds on.","marker":"[36]"},{"why":"Source of the Multi-Queue Fair Queuing algorithm and its fairness theorem that MQFQ-Sticky adapts.","marker":"[40]"},{"why":"Provides the anticipatory scheduling idea for avoiding deceptive idleness, translated into queue keep-alive.","marker":"[43]"},{"why":"Serves as the state-of-the-art GPU multiplexing baseline whose fair-SJF policy MQFQ-Sticky is compared against and outperforms.","marker":"[60]"},{"why":"Supplies the Azure serverless trace used to build realistic heavy-tailed workloads for the latency evaluation.","marker":"[71]"},{"why":"Justifies the long-tailed reuse-distance assumption behind the per-function TTL keep-alive design.","marker":"[33]"},{"why":"The FaaS control plane in which MQFQ-Sticky is implemented and evaluated.","marker":"[32]"},{"why":"Documents CUDA Unified Virtual Memory, the mechanism underlying the prefetch and swap memory management.","marker":"[5]"},{"why":"Shows the LD_PRELOAD shim approach that MQFQ-Sticky's CUDA interposition is modeled on.","marker":"[12]"}],"fun_headline_variants":["Fair queueing cuts GPU serverless latency up to 20x","GPU FaaS gets fair queuing, latency drops up to 20x","Black-box GPU scheduling via fair queuing, up to 20x speedup","Scheduling GPU functions like I/O yields up to 20x gains","MQFQ-Sticky fair queueing cuts GPU latency up to 20x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design assumes that any black-box CUDA function can be run through a shim that swaps its graphics-memory allocations for unified virtual memory with negligible overhead and identical results; functions that bypass that swap are not covered.","fun_headline_variants_meta":{"raw":{"variants":["Fair queueing cuts GPU serverless latency up to 20x","GPU FaaS gets fair queuing, latency drops up to 20x","Black-box GPU scheduling via fair queuing, up to 20x speedup","Scheduling GPU functions like I/O yields up to 20x gains","MQFQ-Sticky fair queueing cuts GPU latency up to 20x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001005,"raw_usage":{"total_tokens":4269,"prompt_tokens":985,"completion_tokens":3284,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":3184}},"tokens_in":601,"tokens_out":3284,"duration_ms":27843,"temperature":1.0,"reasoning_tokens":3184,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:08:33.842609+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a workload of CUDA functions that link the CUDA runtime statically or allocate memory through low-level interfaces the shim does not intercept, and check whether they run correctly and keep the reported 2x-20x gains; one such function failing or showing the kind of 30% slowdown measured for Srad would break the black-box claim.","supporting_citations":[{"cited_title":"In Proceedings of the 52nd International Conference on Parallel Processing","cited_arxiv_id":null,"evidence_quote":"Source of the Multi-Queue Fair Queuing algorithm and its fairness theorem that MQFQ-Sticky adapts."},{"cited_title":"2019.{Multi- Queue} Fair Queuing","cited_arxiv_id":null,"evidence_quote":"Provides the anticipatory scheduling idea for avoiding deceptive idleness, translated into queue keep-alive."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as the state-of-the-art GPU multiplexing baseline whose fair-SJF policy MQFQ-Sticky is compared against and outperforms."}],"review_version":1}