{"id":"9e299ca1-036f-4c19-b748-f6f639c757c1","arxiv_id":"2601.07600","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"On NVIDIA GPUs, MIG partitions provide the strongest temporal isolation for concurrent inference, and Green Contexts approach that level on edge devices only when power throttling is removed.","lead":"This paper benchmarks NVIDIA's GPU isolation mechanisms—MPS, MIG, and Green Contexts—for running multiple deep-learning inferences with predictable timing on A100 and Jetson edge devices. It finds MIG gives strong isolation, while Green Contexts can approach MIG's guarantees only when power is not a limiting factor.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's maximum-stable-IMS f is used as the isolation deadline without a no-contention baseline, so timeout percentages in Figs. 8–11 may conflate inherent latency variability with isolation failure, undermining the GC-near-MIG claim.","rationale":"The reader's weakest_assumption identifies Algorithm 1's f as potentially conflating baseline instability with isolation failure; I agree that this is the most load-bearing concern. The central claim is comparative and quantitative, and every timeout percentage in Figures 8–11 is computed against deadlines that derive from f. If f is a run-specific fit with unknown statistical properties, then the reported percentages have no controlled baseline, and the gap between GC on AGX and on Nano could be an artifact of different slack. The proposed concrete test—measuring isolated timeout rates at f, and re-estimating f via a high quantile—directly tests whether the observed results are due to isolation or to the choice of deadline. This does not change the reader's CONDITIONAL verdict; it strengthens the rationale for the condition: the authors should provide baseline-timeout data and a statistically grounded deadline before the comparison can be accepted. I did not identify a more damaging concern: the cross-platform comparison to MIG is qualitative, but the paper is careful to note it is not comparing absolute inference times, so the absence of a direct MIG baseline on AGX is a limitation rather than the central threat. The f issue is more fundamental because it affects the validity of every quantitative conclusion, including the A100 MIG results. The paper's contribution—a first comparative evaluation of GC—remains useful, but the support for the headline claim is conditional on the deadline-setting methodology being shown to be sound or at least not biased across technologies.","tokens_in":13103,"tokens_out":5840,"duration_ms":63529,"concrete_test":"Run each fixed process at its Algorithm-1 f in complete isolation for the same number of inferences (or wall-clock duration) as used in the contention experiments, repeated at least 10 times, and record the timeout percentage with confidence intervals. If the isolated timeout rate is non-zero, or if its confidence interval overlaps with the rate observed under contention for GC on AGX or MIG on A100, then Figures 8–11 conflate baseline instability with isolation failure. Additionally, recompute f from a long latency trace using a high quantile (e.g., the 99.9th percentile) instead of the zero-violation empirical search, and re-run the key contention comparisons; if the ordering of technologies (e.g., GC vs. MPS on AGX) changes, the conclusion is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that Green Contexts approach MIG isolation when power is not limiting—rests on timeout percentages measured against deadlines derived from the maximum stable inference frequency f found by Algorithm 1 (Section IV.C, Eq. 1). Algorithm 1 certifies f only by the absence of violations in N=1000 inferences plus K=3 validation batches. This is a finite-sample order-statistic estimate of the latency tail, not a statistically validated worst-case rate. If the true violation probability at f is nonzero, say 1e-3, then over a contention run of 10^4 inferences one expects ~10 baseline timeouts even with perfect isolation. The paper never reports the timeout rate of the fixed process running alone at its f. Without this baseline, the low timeout percentages for GC on AGX (Fig. 10) and the high ones on Nano (Fig. 9) cannot be cleanly attributed to isolation properties or power throttling; they may instead reflect different amounts of slack in f across platforms/technologies. Moreover, since f is used for every technology, the cross-technology comparison (Figs. 8, 11) is quantitatively unreliable if f has different conservatism across MIG, MPS, and GC. This is the load-bearing weak point: the entire quantitative evaluation of isolation is filtered through an unvalidated, run-specific threshold.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares NVIDIA GPU isolation mechanisms—MPS, MIG, and Green Contexts (GC)—for providing predictable DNN inference times in safety-critical settings. The methodology has three parts: a partitioning-overhead study on the A100 and Jetson Orin Nano; a calibration stage (Algorithm 1) that determines a 'maximum stable inference frequency' f per model and technology; and contention experiments in which one process runs at f while a second (or several) processes increase their request rate, with timeout percentages used as the isolation metric. The main conclusions are that MIG gives the strongest temporal isolation, MPS gives moderate improvement, GC is not reliable on the power-constrained Orin Nano, and GC can approach MIG-like isolation on the Orin AGX when power is not limiting.","tokens_in":13408,"tokens_out":8609,"duration_ms":81404,"significance":"If the conclusions hold, the paper would provide a valuable empirical data point: Green Contexts, a new fine-grained SM-partitioning mechanism, could serve as a low-overhead, more flexible substitute for MIG-like isolation on power-unconstrained edge GPUs. The identification of power throttling as the cause of GC's poor isolation on the Orin Nano is also practically useful. The paper covers a timely technology and spans multiple platforms and model classes. However, the quantitative evidence for the headline 'GC approaches MIG' claim is currently weaker than the text suggests: the deadline f is not statistically validated, no solo-running baseline is reported, and the timeout figures contain no uncertainty information. The conclusions may be correct, but the evidence as presented does not yet meet the standard needed for safety-critical timing claims.","major_comments":[{"comment":"The deadline f for the fixed process is certified only by absence of violations over N=1,000 inferences plus K=3 validation batches, and the INCREASE/DECREASE step sizes are unspecified. This is a finite-sample heuristic, not a validated worst-case rate. No control experiment reports the timeout percentage of each fixed process running alone at f. As a result, the timeout percentages in Figs. 8–11 conflate inherent latency variability with isolation failure. Concretely, if the true violation probability at f were 10^-3, the 3,000-inference validation would pass with ~5% probability, but a 10^4-inference contention run would show ~10 timeouts even under perfect isolation. Since f is recomputed per technology, any difference in conservatism across standalone/MPS/MIG/GC biases the cross-technology comparisons. Please add solo-baseline timeout rates at f for each platform/technology/model, o","section":"Section IV.C, Algorithm 1, Eq. (1)"},{"comment":"The central measurements are presented as 'trend' lines with no confidence intervals, number of repetitions, or per-point number of inferences. Timeout percentage is a binomial proportion; its uncertainty is large when the denominator is small. Without this information, the text's distinctions — 'close to zero' (Fig. 8c), 'minimal' (Fig. 8c), 'drastically reduced' (Fig. 10) — cannot be assessed, and the 'near MIG' claim is not quantitatively supported. Please report the number of inferences per IMS point, the number of independent runs, and error bars or confidence intervals for every curve.","section":"Section V.D, Figs. 8–11"},{"comment":"The headline claim that GC 'can deliver levels of isolation approaching those of MIG' is a cross-platform qualitative comparison: MIG is not available on Orin, so the comparison is to A100 MIG results on different hardware, with GPU frequency fixed at 1.02 GHz. The paper's own observation that GC lacks memory isolation (Section III.C and Future Work) further limits the generalization to memory-bound mixed workloads. The conclusion should be qualified as compute-resource isolation under the tested power-unconstrained configuration, not full temporal isolation comparable to MIG.","section":"Section V.D.3, Fig. 10, Conclusions"}],"minor_comments":[{"comment":"The title as printed is misspelled: 'Peformance' should be 'Performance'.","section":"Title"},{"comment":"Pseudocode typos ('RUNTIMEDBATCH', 'valid←false' split) should be corrected. More importantly, the step sizes for INCREASEFREQUENCY and DECREASEFREQUENCY should be stated, since they affect the reproducibility of f.","section":"Algorithm 1"},{"comment":"Define IMS in the captions and state whether the plotted lines are pointwise means or fitted trend lines; the legends only say 'trend analysis'.","section":"Figures 8–11"},{"comment":"Clarify the CUDA/JetPack versions: the A100 uses CUDA 12.1 while Green Contexts require CUDA 12.4; state the exact driver versions used for MPS, MIG, and GC experiments.","section":"Section IV.A"},{"comment":"The motivation emphasizes ensembles of diverse models, but the contention tests use the same network for both processes. A sentence acknowledging this limitation and, ideally, one mixed-model experiment would strengthen the applicability of the conclusions.","section":"Section V.D"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical evaluation of commercial GPU mechanisms, not a new systems contribution; it may still fit a systems/OS venue if scope permits experimental studies. I found no circularity in the derivation; the core problem is evidentiary. The missing solo baseline and absent confidence intervals are fixable with additional experiments, so I recommend major revision rather than rejection. An artifact release would also help reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is the first paper I've seen that actually measures Green Contexts on Jetson hardware for temporal isolation, and that alone makes it worth a look for anyone working on safety-critical inference at the edge. The A100 MIG/MPS results are consistent with prior work, and the Orin Nano vs AGX comparison is the useful part — GC looks weak when power is capped and much closer to MIG when it isn't. That power-throttling interaction is a real, non-obvious finding.\n\nWhat the paper does well: six models across three platforms, TensorRT on the Jetsons, and power/frequency monitoring that gives a plausible mechanism for the Nano failure mode. The authors are also honest about GC's lack of memory isolation and about MIG's rigidity. The writing is clear, and the experimental scope is appropriate for a first characterization.\n\nThe soft spots are real, but they are the usual empirical-paper gaps rather than fatal flaws. The timeout percentages in Figures 8–11 are trend lines with no error bars, no repetition counts, and no per-run variance. That matters because a 5% difference between MPS and GC could easily be noise. More structurally, the deadlines come from Algorithm 1's \"maximum stable inference frequency\" f, which is found by requiring zero violations in 1000 inferences plus three validation batches. That is a finite-sample estimate of the latency tail, not a guarantee. The paper never reports the timeout rate of the fixed process running alone at f, so a nonzero baseline violation rate is conflated with isolation failure. The stress-test note is right about this: the cross-technology comparison is quantitatively unreliable unless f has the same conservatism everywhere, which is not shown. The power-throttling explanation for the Nano is also inferred from monitoring traces rather than demonstrated with a controlled power-cap ablation.\n\nThat said, the central claim survives my reading. It's not circular, the measurements are independent of the conclusions, and the AGX result — GC approaching MIG-like isolation when power headroom exists — is consistent with the monitoring data. The missing baseline and error bars limit confidence, but they don't break the paper.\n\nThe right call is to send this to peer review. A serious referee can push for raw data, repetition statistics, a no-contention baseline, and ideally an explicit power-cap experiment on the Nano. I'd bring it to a reading group focused on GPU resource isolation or real-time inference. If I were working on edge GPU scheduling, I'd cite it as the first GC characterization, even with the caveats.","headline":"First comparative data on NVIDIA Green Contexts for edge inference isolation; the central finding is plausible but the quantitative support is weaker than the prose suggests.","tokens_in":13878,"tokens_out":1296,"would_cite":true,"duration_ms":15965,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that NVIDIA's Green Contexts can give edge GPUs near-MIG temporal isolation when power is not the bottleneck, making them a viable substitute for MIG on power-unconstrained edge devices.","keywords":["GPU isolation","Green Contexts","MIG","MPS","temporal predictability","edge inference","Jetson Orin","safety-critical systems"],"falsifier":"Re-run the Orin Nano contention experiment with the GPU clock locked to a constant frequency (bypassing DVFS) while keeping the same 4-SM Green Context partitions; if timeouts persist, the isolation failure is not caused by power throttle. Alternatively, compute Algorithm 1's maximum stable frequency ten times on the same model and check whether the spread in f predicts the timeout percentages better than the contention level does.","tokens_in":13004,"feed_emoji":"⚡","tokens_out":4055,"duration_ms":39128,"temperature":0.7,"pith_summary":"The paper sets out to determine which GPU isolation mechanism can give safety-critical deep-learning inferences predictable timing when several processes share one GPU. It compares NVIDIA's three mechanisms — MPS, MIG, and Green Contexts — on a server A100 and on Jetson Orin edge boards. Its central finding is that MIG offers the strongest temporal isolation, but on edge hardware Green Contexts, which partition the GPU at the granularity of individual streaming multiprocessors, can approach MIG-level isolation when the board has spare power headroom. On power-constrained boards such as the Orin Nano, power saturation causes GPU frequency throttling that destroys Green Contexts' timing guarantees. The paper concludes that Green Contexts are a promising MIG alternative for edge systems, provided memory isolation is added and power limits are respected.","feed_headline":"Green Contexts nearly match MIG isolation on edge GPUs","feed_subtitle":"Fine-grained SM partitioning keeps inference deadlines when power is available; throttling breaks them on the Orin Nano.","key_machinery":"The central objects are the three isolation mechanisms: MPS (a software layer that merges CUDA contexts to reduce switching overhead), MIG (hardware partitioning into GPU processing clusters, GPCs), and Green Contexts (software-defined CUDA contexts that cap the number of streaming multiprocessors, SMs, a process can use). The paper's measuring device is Algorithm 1, which searches for the maximum stable inference frequency per model and partition, then records the timeout percentage of a fixed-rate process under contention. This converts 'isolation' into an empirical quantity: a technology isolates well if the fixed process never misses its inference deadline as the competing load rises.","core_discovery":"Using a timeout-based benchmark where one process performs inference at a frequency derived by a search algorithm (Algorithm 1) while a competing process increases its request rate, the paper measures isolation as the percentage of timed-out inferences. On the Jetson Orin AGX with the GPU clock fixed at 1.02 GHz, Green Contexts with 4-SM partitions keep timeouts near zero even with up to four concurrent processes, matching the behavior of MIG on the A100. On the Orin Nano, the same setup produces frequent timeouts because two processes together approach the board's ~20 W power limit, causing the GPU to drop its frequency by roughly half. The paper attributes the difference to power headroom,","pith_inferences":["The paper's method of deriving the maximum inference frequency with Algorithm 1 makes the isolation claim contingent on that frequency being a stable property; a natural extension would measure the spread of f across repeated runs and use a distributional deadline instead of a point value.","If the power-throttling explanation is right, then a testable prediction follows: capping the Orin Nano's power budget or reducing the number of concurrently active SMs should restore Green Contexts' isolation without moving to a larger board.","The two-process, identical-model contention setup is a first step; real ensembles mix models, so mixed-model contention (e.g., a memory-bound and a compute-bound network sharing the same SM partition) is the next experiment that would stress test the conclusions.","The finding that Green Contexts show no overhead suggests a path toward dynamic SM reallocation: if memory isolation is added, Green Contexts could enable schedulers that resize partitions at runtime — something MIG cannot do."],"forward_implications":["On power-unconstrained edge GPUs, Green Contexts can serve as a drop-in MIG-like isolation layer with finer granularity than MIG and no measurable throughput overhead.","On power-constrained boards, any isolation mechanism that allows concurrent processes to collectively saturate the power budget will fail to guarantee inference deadlines, because DVFS throttling overrides partitioning.","Temporal stability depends on model type: compute-bound models tolerate resource shrinking better than memory-bound models, so isolation requirements are model-dependent.","MIG remains the strongest isolation mechanism but its static, non-resizable partitions limit adaptability to dynamic real-time workloads, motivating software alternatives.","Existing isolation mechanisms still lack memory isolation for Green Contexts and complete temporal guarantees in all scenarios, so fully predictable shared-GPU inference is not yet solved."],"fun_headline_variants":["Green Contexts match MIG isolation on AGX, fail on Orin Nano","Power throttling breaks Green Contexts isolation on edge boards","Fine-grained SM isolation works until power limit hits on Orin Nano","Green Contexts rival MIG on edge, but power shortage causes timeouts"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The measured isolation numbers assume the per-model deadline fixed by Algorithm 1 is a trustworthy worst-case inference rate; if that deadline already contains run-to-run variability, the reported timeout percentages partly reflect benchmark noise rather than isolation failures, and the same applies to the assumption that two copies of the same network capture the contention behavior of real ensembles.","fun_headline_variants_meta":{"raw":{"variants":["Green Contexts match MIG isolation on AGX, fail on Orin Nano","Power throttling breaks Green Contexts isolation on edge boards","Fine-grained SM isolation works until power limit hits on Orin Nano","Green Contexts rival MIG on edge, but power shortage causes timeouts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1207,"prompt_tokens":635,"completion_tokens":572,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":379,"completion_tokens_details":{"reasoning_tokens":506}},"tokens_in":379,"tokens_out":572,"duration_ms":6453,"temperature":1.0,"reasoning_tokens":506,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T11:00:55.025278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Orin Nano contention experiment with the GPU clock locked to a constant frequency (bypassing DVFS) while keeping the same 4-SM Green Context partitions; if timeouts persist, the isolation failure is not caused by power throttle. Alternatively, compute Algorithm 1's maximum stable frequency ten times on the same model and check whether the spread in f predicts the timeout percentages better than the contention level does.","supporting_citations":[],"review_version":1}