Pith. sign in

REVIEW 4 major objections 6 minor 75 references

Interactive world models pin multi-gigabyte GPU state that cannot be rebuilt or approximated; WorldMove relocates that state under a bit-identity contract so fleets can schedule sessions instead of devices.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 12:05 UTC pith:AE5N6YWY

load-bearing objection Real systems work: bit-exact live migration of window-rewrite world-model caches, with hard numbers and an honest unbuilt composition gap. the 4 major comments →

arxiv 2607.10389 v1 pith:AE5N6YWY submitted 2026-07-11 cs.DC cs.LG

Stateful Worlds, Stateless Elasticity: Exact-State Serving for Interactive World Models

Jin Li (Harvard University) , Jiawei Chen This is my paper
classification cs.DC cs.LG
keywords interactive world modelsexact-state migrationGPU attention cachelive session elasticityadmissibility conditionbit-exact servingincast admission controlverification scheduling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

A live interactive world model keeps a multi-gigabyte attention cache on the GPU that serves it, rewriting most of that cache every generation step. That state cannot be recomputed inside interactive time or approximated without changing the world, so today one user pins one GPU for the whole session. This paper treats the pin as a scheduling problem. WorldMove moves the live cache under one fail-closed guarantee: the destination is bit-identical to the source, or nothing is installed. Same-node moves finish in 18.8 ms; on a 100 Gb fabric the verified transfer fits inside one interactive block, and a session that is still generating converges at a block boundary so the destination continues the world bit for bit. An admissibility condition decides each move: finish inside the readout horizon, over bandwidth that covers the state plus its dirty rate. Lifted to a fleet test, that condition governed 48 of 48 bit-identical consolidations across two providers. The paper also shows verification is a second scheduling plane: receive-path checksums and unscheduled incast can break deadlines even when every delivered byte is correct.

Core claim

Exact continuity for interactive world models forces whole-state motion of a heavily mutating, readout-irreducible GPU attention cache. That motion is admissible exactly when the move finishes inside the contract horizon and fabric bandwidth covers state size plus dirty rate. WorldMove realizes the motion fail-closed (bit-identical destination or nothing installed), and the same condition plus incast-aware admission make fleet consolidation of live sessions a measurable schedulability problem rather than a permanent device pin.

What carries the argument

The Exact-State Admissibility Condition: a migration is legal only if T_migrate(S) < H and B_fabric ≥ S/H + D (finish inside the readout horizon over bandwidth covering resident state plus dirty rate). It decides each move and lifts to a fleet response-time test for non-preemptive atomic transfers.

Load-bearing premise

No lossy reduction of the cache preserves the exact world, so the only interactive-time path to continuity is moving the whole state inside one GPU architecture.

What would settle it

Find a production window-rewrite world model that resumes a contract-indistinguishable trajectory after lossy compression, cross-architecture recompute, or partial-state transfer inside one interactive block; that would collapse the forced whole-state motion premise.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Live world sessions can be consolidated, evacuated, and rebalanced without breaking bit-exact continuity.
  • Bit-exact resume holds only inside one GPU architecture, so architecture-aware placement is mandatory for exact contracts.
  • At datacenter rates, verification placement—not wire bandwidth alone—sets how many concurrent migrations a receiver can admit.
  • A lossless GPU codec widens the admissible fabric region so fabrics too slow for raw motion become legal.
  • Incast-aware admission can hold zero deadline misses to 1.4× offered load and shed overload as explicit rejects.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • As per-session context outgrows HBM growth, any window-rewrite generative engine—not only video world models—will hit the same forced-motion regime.
  • Commodity KV movers that check only arrival, not end-to-end identity, would install silent corruptions under the contracts this paper measures.
  • Composing the live serving loop with the verified line-rate mover on one fabric is the remaining step that would turn the separate exhibits into a single deployable control plane.
  • The same horizon-plus-dirty-rate gate may apply to any real-time system whose working set rewrites most of itself every quantum and cannot be approximated.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that interactive world-model sessions pin GPUs because their multi-GB attention caches are readout-irreducible under an exact-continuity contract, and presents WorldMove/Worldline as the primitive and substrate that make that pin a schedulable resource. WorldMove relocates the live cache under a fail-closed bit-identity guarantee (destination bit-identical or nothing installed), measuring 18.8 ms same-node (101× save/load) and checksum-verified 92.1–94.8 Gb/s on a 100 Gb fabric so the 1.67 GB movable set fits inside one interactive block; live pre-copy converges at a block boundary with bit-identical continuation. An admissibility condition T_migrate(S)<H and B_fabric ≥ S/H+D decides legal moves and lifts to a fleet test that governed 48/48 bit-identical consolidation migrations. The paper further shows verification placement and unscheduled incast as a second scheduling plane, with an incast-aware controller holding zero misses to 1.4× load, and a lossless GPU codec widening the admission gate. Serving loop and mover are exercised end-to-end separately; their composition on one 100 Gb fabric is explicitly unbuilt.

Significance. If the measurements and construction hold, this is a timely systems contribution for an emerging serving regime: fixed-shape diffusion quanta, heavily mutating multi-GB resident state, and no safe approximate resume. Strengths include concrete kill-family and dirty-structure measurements (ρ_WWS≈0.71–0.78), bit-exact live migration against active generation with independent oracles and negative controls, line-rate verified transfer within ~6% of the verify-off ceiling, 48/48 executed fleet migrations across providers, and a careful characterization of receive-path verification and incast as admission-control problems rather than pure transport. The paper is unusually explicit about scope (unbuilt composition, architecture-bounded T0, simulation beyond the executed loop). The admissibility condition and verification-plane findings would be useful even if later engines admit additional reduction handles.

major comments (4)
  1. [§3, §4 motion criterion] §3 and the motion criterion of §4: the forced-motion premise (and thus the structural status of B ≥ S/H+D) rests on the kill-family exhausting contract-preserving reduction for window-rewrite engines. The measured 60–92 dB divergences for eviction, quantization, low-rank projection, and cross-session dedup on Matrix-Game-2 are strong against those handles, but do not rule out model-aware residual codecs, learned compressors trained under R, or other architecture-portable resumes that still meet H. The paper already frames necessity as systems rigor rather than formal proof; the load-bearing text should state the claim as “for the handles evaluated, and under the three-escape test as stated” and treat any stronger universality claim as a conjecture with a concrete falsification path (e.g., a residual codec that keeps R inside 400 ms).
  2. [§6 Evaluation scope, §8] §6 and §8: the abstract and conclusion present exact-state elasticity as a joint scheduling problem over transport and verification, yet “their composition on one fabric is unbuilt” and “we have not run the live serving loop over the 100 Gb fabric.” The 48/48 loop, live-chase, and line-rate cells are separately strong, but end-to-end fleet claims (admission under concurrent generation + verified motion + incast control) currently rest on composition by argument plus simulation beyond the executed loop. Either complete a minimal composed cell or systematically demote joint-scheduling claims to “separately validated components + projected composition,” with a clear table of demonstrated vs projected results.
  3. [§3.3, §6, §8 Backend breadth] §3 and §6: quantitative spine is one backend family (Matrix-Game-2) plus a calibration row on open-oasis and a model-blind LLM KV demo. Dirty fraction, kill-family, and quality-horizon drift (~10 s) are therefore regime-class claims on thin backend breadth (N≤2). The update-rule classifier (§3.3) is the right transfer device, but the paper should either add a second window-rewrite engine under the same instruments or more sharply bound which constants (S, ρ_WWS, D, drift) are expected to transfer vs which are MG2-specific operating points.
  4. [§4 Eqs. (1)–(3), §6.3] §4 Eq. (1)–(3) and fleet test: the fluid bound N ≤ HB/C is acknowledged to overcount (footnote: ~57 ms intercept; contended T(2)=415 ms > 400 overrides), and live dirty is “measured per-move but not yet inside a serialized fleet schedule.” For the schedulability claim that governed consolidation, the paper should lead with the measured response-time structure and incast-aware controller results, and present Eq. (3) strictly as a homogeneous fluid upper bound with stated blocking/interference terms (Appendix C), not as the operational admission integer without those caveats in the main text.
minor comments (6)
  1. [Figure 2] Figure 2 regime map is useful; label measured anchors vs projections more explicitly in the caption (e.g., 10 Hz MG2, 800 Gb ports).
  2. [Table 2] Table 2 mixes code-path, fabric, live-cell, and cold-rebuild rows; a one-line note that cold reopen (260–322 ms) is not an exact-class competitor would reduce misreading against the 18.8 ms claim.
  3. [§4] Notation: H is both readout horizon and (elsewhere) host; B is bandwidth and sometimes block. A short symbol table would help.
  4. [§6.1, Appendix G] Appendix G cross-architecture readout (median 61 dB PSNR) is important for the T0/T1 boundary; a one-sentence pointer in §6.1 to the excluded block-0 transient would avoid over-reading the floor.
  5. [§7] Related work: TurboServe and Llumnix comparisons are fair; ensure arXiv dates/versions cited match the concurrent-work framing so priority claims stay accurate.
  6. [Appendix J, §6] Artifact statement (Appendix J) is welcome; if the journal process allows, link the numbers manifest and decide.py grid in the main evaluation intro for reproducibility.

Circularity Check

0 steps flagged

No significant circularity: empirical systems paper whose admissibility gate, fleet lift, and mover claims are measured or elementary, not forced by definition or self-citation.

full rationale

The load-bearing chain is (i) kill-family measurements that lossy reduction breaks readout R, (ii) the elementary exact-state admissibility condition T_migrate(S)<H and B_fabric ≥ S/H+D with S,D,H,B measured or contract-declared, (iii) a standard non-preemptive deadline-monotonic fleet lift, and (iv) end-to-end bit-identity oracles plus negative controls on WorldMove. None of these reduces to its own inputs by construction: the inequality is stated as elementary necessity under the contract, sufficiency is a constructive system that commits only on fingerprint match, and fleet/consolidation results are executed migrations (48/48) or Engset/repairman sizing driven by measured costs rather than defining success as the model output. The coupon-collector dirty-set recursion is a standard occupancy formula calibrated on dirty structure and pre-registered against a different target (convergence boundary); that is model validation, not a fitted quantity renamed as prediction. There is no uniqueness theorem imported from the authors, no ansatz smuggled via self-citation, and no self-definitional loop between the gate and the reported rates. The paper is self-contained against its own bit-identity and deadline oracles; residual concerns (incomplete kill-family, uncomposed 100 Gb serving+mover loop) are scope/necessity risks, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 4 invented entities

The central claim rests on a contract choice (exact continuity as bit identity at tier T0), empirical workload facts (irreducible multi-GB window-rewrite cache, high dirty rate), and systems constructions (WorldMove, Worldline, admission controller). Free parameters are mostly measured constants used as inputs to the gate, not fitted to force the headline. Invented entities are engineering artifacts and named measurement categories rather than unobserved physical mediators.

free parameters (4)
  • readout horizon H (C1 block period)
    Contract parameter declared per class; paper uses 400 ms (and 600 ms in some cells). Not fitted to prove migration works, but the admissible region is defined relative to this choice.
  • movable state size S and dirty rate D
    Measured on Matrix-Game-2 (S≈1.67 GB movable, ρ_WWS≈0.71–0.78, D≈0.93 GB/s at 0.78 Hz). Treated as fixed inputs to the admissibility gate; results are specific to these measured constants and the window-rewrite class.
  • prewarm hit-rate break-even λ*
    Consolidation economy break-even ≈0.64 (25–75% band [0.55,0.73] over 64 seeds) is estimated from measured hot vs first-touch costs; it shapes the reclaim claim rather than the bit-exact mover claim.
  • CRC engine capacity B_crc and timeout exponents
    Verification-plane fan-in bound uses measured 52.6 GB/s 32-thread CRC and NIC local-ACK timeout quanta; placement conclusions depend on these stack-specific rates.
axioms (5)
  • domain assumption Exact-continuity contract at tier T0 requires byte equality of resident state for a legal resume; approximate or statistical continuity is a different contract class.
    Stated throughout §1–§4; drives the kill-family and fail-closed commit design.
  • domain assumption The multi-GB attention cache cannot be recomputed from prompt/history inside the interactive readout horizon for the targeted engines.
    Core premise of the pin; supported by timing arguments and motion criterion in §4, not a formal lower bound for all models.
  • domain assumption Native-rate bit-exact re-execution does not hold across GPU architectures even with matched software stacks.
    Measured in §6.1 (A100×H100, A100×L40S); used to argue motion beats re-derivation across architecture boundaries.
  • ad hoc to paper A started exact-state move is modeled as atomic non-preemptive work to commit or abort; partial delivery earns nothing under T0.
    Scheduling model in §4; standard for fail-closed install but stronger than classical pre-copy VM assumptions.
  • standard math Deadline-monotonic response-time analysis specialized with blocking/interference terms applies to concurrent atomic state moves on a shared channel.
    Appendix C / §4 fleet test citing classical non-preemptive real-time scheduling; fluid N≤HB/C is an approximation they note is not tight.
invented entities (4)
  • WorldMove / Worldline session object independent evidence
    purpose: Bit-exact flatten/transfer/verify/commit primitive and thin six-verb control plane for contract-bearing sessions.
    Primary systems artifact; independent evidence is the measured migrations and API behavior reported in the paper/artifact.
  • Exact-state admissibility condition (T_migrate < H and B ≥ S/H + D) independent evidence
    purpose: Decide legal migrations and lift to fleet schedulability for readout-irreducible resident GPU state.
    Named criterion combining classical deadline/flux ideas with this object’s constants; validated against measured envelopes rather than derived from first principles alone.
  • Kill-family of lossy handles no independent evidence
    purpose: Classify reduction techniques shown to break the divergence contract for this state.
    Measurement taxonomy local to the paper’s engines; purpose is to close reduction escapes.
  • Readout-irreducible resident GPU state (window-rewrite class) no independent evidence
    purpose: Name the resource that must be moved whole under exact continuity.
    Workload characterization entity; membership classified by update-rule dirty measurements (§3.3).

pith-pipeline@v1.1.0-grok45 · 33934 in / 4369 out tokens · 51569 ms · 2026-07-14T12:05:45.296347+00:00 · methodology

0 comments
read the original abstract

A persistent interactive world model keeps its running state resident on the GPU that serves it: a multi-gigabyte attention cache, almost all of it rewritten at every generation step. That state cannot be recomputed in interactive time or approximated without changing the world, so a live session pins its device. The pin is a scheduling problem. WorldMove moves a live session under one guarantee: the destination is bit-identical to the source, or nothing is installed. It relocates the cache in 18.8 ms same-node, 101x faster than save/load. It holds a checksum-verified 92.1-94.8 Gb/s on a 100 Gb fabric. At that rate the cache fits inside one interactive block. Migrating an actively generating session, it converges at a block boundary and the destination continues the world bit for bit. An admissibility condition decides each move. The move must complete inside the readout horizon, over bandwidth that covers the state plus its dirty rate. Lifted to a fleet schedulability test, it governed a consolidation loop that executed 48 of 48 migrations bit-identical across two providers. Two constraints are structural. Bit-exactness survives only inside a controlled configuration of one GPU architecture, so moving the state is the only way to preserve it exactly in interactive time. Verification cannot hide inside the wire on this fabric. Receive-path checksums stall the transport at protocol timescales under fan-in, and unscheduled incast silently collapses a receiver while every delivered byte stays correct. An incast-aware admission controller holds zero misses to 1.4x offered load and sheds overload as rejects. A lossless GPU codec widens the admission gate to fabrics raw motion cannot use. We exercise the serving loop and the mover separately, each end to end. Their composition on one fabric is unbuilt. Exact-state elasticity is a joint scheduling problem over transport and verification.

Figures

Figures reproduced from arXiv: 2607.10389 by Jiawei Chen, Jin Li (Harvard University).

Figure 1
Figure 1. Figure 1: Worldline: a thin control plane over a state-motion data plane whose commits are receipt-checked. One admitted move: admission and placement (1), flatten (2), transfer (3), pipelined CRC on a lane with its own capacity (4), fail-closed commit on receipt match (5); rejected moves fall to the prewarm pool. p50). Telemetry is fail-closed. Every run fingerprints the host, asserts collector liveness within 15 s… view at source ↗
Figure 2
Figure 2. Figure 2: The serving-regime map: each object a point (dirty rate 𝐷 vs bandwidth 𝐵). Left of the gate migration misses the deadline; above the 𝐷=𝐵 phase line pre-copy cannot converge. MG2 (𝜌WWS≈0.75) crosses regimes as step rate scales 0.78 → 10 Hz; append-only objects never leave. node at 54Gb/s) with uniform receivers, so the naive 57/72 miss was pure incast. This collapse and the scheduled contention law are the … view at source ↗
Figure 3
Figure 3. Figure 3: Two concurrent full-cache verified moves through one receiver (barrier-synchronized incast, 512 MB chunks): in collapsed repetitions both flows freeze synchronously for 1.7–1.8 s; the no-op-executor control is stable. engineer away, while the loss here is induced by receiver-side verification compute, a trigger that literature does not cover. Second, checksum capacity caps the fan-in. A receive path that c… view at source ↗
Figure 4
Figure 4. Figure 4: Verified-transfer price list, measured on one fabric on one day (two CloudLab d6515s, 100GbE, 2026-07-05): with off-path pipelined verification on by default, verified throughput reaches 92.09 Gb/s at uniform 64 MB chunks (94% of the 97.95Gb/s wire ceiling) and 94.8Gb/s with the tail schedule. unstable, stable, and readout-orthogonal directions alike collapse fidelity by ≥60 dB (means 73–92 dB, all 12 prof… view at source ↗
Figure 5
Figure 5. Figure 5: Consolidation phase diagram: sessions per device 𝐾 ∗ under the 400 ms/1% SLO versus think time 𝑍 (machine￾repairman CTMC with measured cost constants; sessions park at 1.67GB cores). 0.0 0.2 0.4 0.6 0.8 1.0 interactive fraction f (continuous-interactive → ) 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 idle-prone duty d 51 40 30 19 9 1 35 28 19 13 7 1 22 18 13 8 3 1 9 8 4 4 2 1 10% 30% representative mix (24%) 0 10 … view at source ↗
Figure 6
Figure 6. Figure 6: Fleet reclaimed at a 1% reactivation SLO bound over interactive fraction 𝑓 and idle-prone duty 𝑑 (𝐶𝑚=96.4 ms): up to 51% in the idle-prone corner, ∼0% at continuous-interactive load. is the AFK tail, the near-idle bottom decile the synthetic assumption erases. The warm buffer is an Erlang buffer of ∼2 device equivalents at 120 sessions. Aggressive thresholds (5 s) generate a reactivation flow two orders of… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 16 linked inside Pith

  1. [1]

    W. Kwon, Z. Li, S. Zhuang, et al. Efficient Memory Management for Large Language Model Serving with PagedAttention. SOSP, 2023

    2023

  2. [2]

    G.-I. Yu, J. S. Jeong, G.-W. Kim, et al. Orca: A Distributed Serving System for Transformer-Based Generative Models. OSDI, 2022

    2022

  3. [3]

    Agrawal, N

    A. Agrawal, N. Kedia, A. Panwar, et al. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. OSDI, 2024

    2024

  4. [4]

    Zheng, L

    L. Zheng, L. Yin, Z. Xie, et al. SGLang: Efficient Execution of Structured Language Model Programs. NeurIPS, 2024

    2024

  5. [5]

    Zhang, Y

    Z. Zhang, Y. Sheng, T. Zhou, et al. H 2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS,

  6. [6]

    Y. Liu, H. Li, Y. Cheng, et al. CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving. ACM SIGCOMM,

  7. [7]

    Hooper, S

    C. Hooper, S. Kim, H. Mohammadzadeh, et al. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization. NeurIPS, 2024. arXiv:2401.18079

    Pith/arXiv arXiv 2024

  8. [8]

    C. L. Liu and J. W. Layland. Scheduling Algorithms for Multipro- gramming in a Hard-Real-Time Environment.Journal of the ACM, 20(1):46–61, 1973

    1973

  9. [9]

    S. Vestal. Preemptive Scheduling of Multi-Criticality Systems with Varying Degrees of Execution Time Assurance. RTSS, 2007, pp. 239–243

    2007

  10. [10]

    J. W. S. Liu, K.-J. Lin, W.-K. Shih, et al. Algorithms for Scheduling Imprecise Computations. IEEE Computer, 24(5):58–68, 1991

    1991

  11. [11]

    J. P. Lehoczky and S. Ramos-Thuel. An Optimal Algorithm for Schedul- ing Soft-Aperiodic Tasks in Fixed-Priority Preemptive Systems. RTSS, 1992, pp. 110–123

    1992

  12. [12]

    L. Sha, R. Rajkumar, and J. P. Lehoczky. Priority Inheritance Protocols: An Approach to Real-Time Synchronization.IEEE Trans. Computers, 39(9):1175–1185, 1990

    1990

  13. [13]

    RTFM: A Real-Time Frame Model

    World Labs. RTFM: A Real-Time Frame Model. worldlabs.ai/blog/rtfm, 2025

    2025

  14. [14]

    Oasis: Interactive AI Video Game Model

    Decart. Oasis: Interactive AI Video Game Model. decart.ai/publications, 2024

    2024

  15. [15]

    Marble: A Multimodal World Model.worldlabs.ai/blog, 2025

    World Labs. Marble: A Multimodal World Model.worldlabs.ai/blog, 2025

    2025

  16. [16]

    S. Yang, W. Huang, R. Chu, et al. LongLive: Real-time Interactive Long Video Generation. arXiv:2509.22622, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Introducing Interactive Video

    Odyssey. Introducing Interactive Video. odyssey.ml/introducing-interactive-video, 2025

    2025

  18. [18]

    D. Riley. Decart raises $100M on $3.1B valuation to grow real-time AI video platform. SiliconANGLE, August 7, 2025. siliconangle.com/2025/08/07/

    2025

  19. [19]

    Oasis (Minecraft clone).Wikipedia, en.wikipedia.org/wiki/ Oasis_(Minecraft_clone) (accessed 2026-07-07; five-minute session cap at launch)

    2026

  20. [20]

    A. Li. Google rolling out ‘Project Genie’ to generate playable worlds. 9to5Google, January 29, 2026. 9to5google.com/2026/01/29/ google-project-genie

    2026

  21. [21]

    High Bandwidth Memory.Wikipedia, en.wikipedia.org/wiki/High_Bandwidth_Memory (accessed 2026-07-02; per-stack capacities by generation; JEDEC HBM4 April 2025)

    2026

  22. [22]

    Qwen2.5 Technical Report (Qwen2.5-0.5B- Instruct)

    Qwen Team, Alibaba. Qwen2.5 Technical Report (Qwen2.5-0.5B- Instruct). arXiv:2412.15115, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Hamdaoui and P

    M. Hamdaoui and P. Ramanathan. A Dynamic Priority Assignment Technique for Streams with (m,k)-Firm Deadlines. IEEE Transactions on Computers, 44(12):1443–1451, 1995

    1995

  24. [24]

    J. Yi, M. Kim, P. H. Cho, et al. WorldKV: Efficient World Memory with World Retrieval and Compression. arXiv:2605.22718, 2026

    Pith/arXiv arXiv 2026

  25. [25]

    Hsieh, Z

    C. Hsieh, Z. Zong, X. Chen, et al. FlowPrefill: Decoupling Preemption from Prefill Scheduling Granularity to Mitigate Head-of-Line Blocking in LLM Serving. arXiv:2602.16603, 2026

    arXiv 2026

  26. [26]

    Y. Zeng, J. Zheng, C. Zheng, et al. X-Cache: Cross-Chunk Block Caching for Few-Step Autoregressive World Models Inference. arXiv:2604.20289, 2026

    Pith/arXiv arXiv 2026

  27. [27]

    Nawaz, A

    U. Nawaz, A. Heakl, U. Khan, et al. WorldCache: Content-Aware Caching for Accelerated Video World Models. arXiv:2603.22286, 2026

    arXiv 2026

  28. [28]

    F. Ye, Z. Li, X. Zhong, et al. GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads. arXiv:2604.04335, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Sheng, et al

    Z. Sheng, et al. SlackServe: Adaptive Resource Management and Quality Control for Streaming Video Generation. arXiv:2606.15319, June 2026

    arXiv 2026

  30. [30]

    Zhao, et al

    Y. Zhao, et al. ReMP: Low-Downtime Runtime Model-Parallelism Reconfiguration for LLM Serving. arXiv:2606.18741, June 2026

    Pith/arXiv arXiv 2026

  31. [31]

    Chen, et al

    H. Chen, et al. ShuntServe: Cost-Efficient LLM Serving on Heteroge- neous Spot GPU Clusters. arXiv:2606.18600, June 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Yuan, et al

    J. Yuan, et al. HEAL: Demystifying Numerical Instability in LLM Inference. arXiv:2606.21023, June 2026

    Pith/arXiv arXiv 2026

  33. [33]

    Jiang, H

    Y. Jiang, H. Wang, H. Bao, et al. TurboServe: Serving Streaming Video Generation Efficiently and Economically.arXiv:2606.19271, 2026

    Pith/arXiv arXiv 2026

  34. [34]

    T. Feng, Z. Li, S. Yang, et al. StreamDiffusionV2: A Streaming Sys- tem for Dynamic and Interactive Video Generation.MLSys, 2026 (arXiv:2511.07399)

    arXiv 2026

  35. [35]

    J. Fang, J. Pan, X. Sun, et al. xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism. arXiv:2411.01738, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Clark, K

    C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. InNSDI, 2005

    2005

  37. [37]

    P. J. Denning. The Working Set Model for Program Behavior.Commu- nications of the ACM, 11(5):323–333, 1968

    1968

  38. [38]

    Kilburn, D

    T. Kilburn, D. B. G. Edwards, M. J. Lanigan, and F. H. Sumner. One-Level Storage System.IRE Transactions on Electronic Computers, EC-11(2):223– 235, 1962

    1962

  39. [39]

    CRIU: Checkpoint/Restore In Userspace.https://criu.org(accessed 2026)

    2026

  40. [40]

    B. Sun, Z. Huang, H. Zhao, W. Xiao, X. Zhang, Y. Li, and W. Lin. Llumnix: Dynamic Scheduling for Large Language Model Serving. In OSDI, 2024

    2024

  41. [41]

    Stoyanov et al

    R. Stoyanov et al. CRIUgpu: Transparent Checkpointing of GPU- Accelerated Workloads.arXiv:2502.16631, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Qin et al

    R. Qin et al. Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot. InUSENIX FAST, 2025

    2025

  43. [43]

    NIXL: NVIDIA Inference Xfer Library

    NVIDIA. NIXL: NVIDIA Inference Xfer Library. github.com/ai-dynamo/nixl, 2025–2026. Point-to-point transfer abstraction for inference frameworks (e.g. Dynamo), with pluggable UCX/GPUDirect/storage backends

    2025

  44. [44]

    Zhong et al

    Y. Zhong et al. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. InOSDI, 2024

    2024

  45. [45]

    Patel et al

    P. Patel et al. Splitwise: Efficient Generative LLM Inference Using Phase Splitting. InISCA, 2024

    2024

  46. [46]

    L. Su. Execution-State Capsules: Graph-Bound Execution-State Check- point and Restore for Low-Latency, Small-Batch, On-Device Physical-AI Serving. arXiv:2606.20537, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    N. Cankaya. Bit-Exact AI Inference Verification Without Performance Tradeoffs. arXiv:2606.00279, 2026. ICML 2026 TAIGR workshop

    Pith/arXiv arXiv 2026

  48. [48]

    W. Xiao, R. Bhardwaj, R. Ramjee, et al. Gandiva: Introspective Cluster Scheduling for Deep Learning. InOSDI, 2018

    2018

  49. [49]

    A. K. Erlang. Solution of Some Problems in the Theory of Probabilities of Significance in Automatic Telephone Exchanges.Post Office Electrical Engineers’ Journal, 10:189–197, 1917

    1917

  50. [50]

    T. O. Engset. Die Wahrscheinlichkeitsrechnung zur Bestimmung der Wählerzahl in automatischen Fernsprechämtern.Elektrotechnische Jin Li and Jiawei (Alexon) Chen Zeitschrift, 39, 1918. (Finite-source loss model; English translation in Telektronikk, 1992.)

    1918

  51. [51]

    Azure LLM Inference Trace (2023, 2024).Azure Public Dataset,github.com/Azure/AzurePublicDataset

    Microsoft Azure. Azure LLM Inference Trace (2023, 2024).Azure Public Dataset,github.com/Azure/AzurePublicDataset

    2023

  52. [52]

    Wang et al

    Y. Wang et al. BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems.KDD, 2025; dataset github.com/HPMLL/BurstGPT

    2025

  53. [53]

    Gandhi, M

    A. Gandhi, M. Harchol-Balter, and I. Adan. Server Farms with Setup Costs.Performance Evaluation, 67(11):1123–1138, 2010

    2010

  54. [54]

    A. L. Scherr.An Analysis of Time-Shared Computer Systems. MIT Press,

  55. [55]

    (The finite-source machine-repairman model.)

  56. [56]

    NVIDIA.CUDA C++ Programming Guide, § Peer-to-Peer Memory Access (IOMMU note), 2026.docs.nvidia.com/cuda/

    2026

  57. [57]

    cuda-checkpoint: Checkpoint and Restore Utility for CUDA Applications (driver-580 migration API and demos)

    NVIDIA. cuda-checkpoint: Checkpoint and Restore Utility for CUDA Applications (driver-580 migration API and demos). github.com/NVIDIA/cuda-checkpoint, accessed 2026-07-07

    2026

  58. [58]

    Phung-Duc

    T. Phung-Duc. Exact Solutions for M/M/c/Setup Queues.Telecommuni- cation Systems, 64(2):309–324, 2017

    2017

  59. [59]

    Szymkowicz

    S. Szymkowicz. Decart’s new world model can simulate hours of photorealistic driving. TechCrunch, June 10, 2026. techcrunch.com/2026/06/10/

    2026

  60. [60]

    Vendor documentation

    NVIDIA GB300 NVL72 / ConnectX-8: 800 Gb/s per-GPU scale-out networking, 2026. Vendor documentation

    2026

  61. [61]

    J. H. Saltzer, D. P. Reed, and D. D. Clark. End-to-End Arguments in System Design.ACM Transactions on Computer Systems, 2(4):277–288, 1984

    1984

  62. [62]

    D. D. Clark, V. Jacobson, J. Romkey, and H. Salwen. An Analysis of TCP Processing Overhead.IEEE Communications Magazine, 27(6):23–29, 1989

    1989

  63. [63]

    Arslan and A

    E. Arslan and A. Alhussen. Fast Integrity Verification for High-Speed File Transfers. arXiv:1811.01161, 2018

    Pith/arXiv arXiv 2018

  64. [64]

    S. Liu, R. Kettimuthu, et al. Towards Optimizing Large-Scale Data Transfers with End-to-End Integrity Verification.IEICE Trans. Inf. & Syst., 2019

    2019

  65. [65]

    Mittal, A

    R. Mittal, A. Shpiner, A. Panda, et al. Revisiting Network Support for RDMA. InACM SIGCOMM, 2018

    2018

  66. [66]

    C. Guo, H. Wu, Z. Deng, et al. RDMA over Commodity Ethernet at Scale. InACM SIGCOMM, 2016

    2016

  67. [67]

    Y. Zhu, H. Eran, D. Firestone, et al. Congestion Control for Large-Scale RDMA Deployments. InACM SIGCOMM, 2015

    2015

  68. [68]

    Vasudevan, A

    V. Vasudevan, A. Phanishayee, H. Shah, et al. Safe and Effective Fine- grained TCP Retransmissions for Datacenter Communication. InACM SIGCOMM, 2009

    2009

  69. [69]

    §12.7.34, Local ACK Timeout (4.096 𝜇s × 2𝑛)

    InfiniBand Trade Association.InfiniBand Architecture Specification, Volume 1, Release 1.3, 2015. §12.7.34, Local ACK Timeout (4.096 𝜇s × 2𝑛)

    2015

  70. [70]

    J. C. Mogul and K. K. Ramakrishnan. Eliminating Receive Livelock in an Interrupt-Driven Kernel.ACM Transactions on Computer Systems, 15(3):217–252, 1997

    1997

  71. [71]

    Jeffay, D

    K. Jeffay, D. F. Stanat, and C. U. Martel. On Non-Preemptive Scheduling of Periodic and Sporadic Tasks. InIEEE RTSS, 1991

    1991

  72. [72]

    Wilson, H

    C. Wilson, H. Ballani, T. Karagiannis, and A. Rowstron. Better Never than Late: Meeting Deadlines in Datacenter Networks. InACM SIG- COMM, 2011

    2011

  73. [73]

    C.-Y. Hong, M. Caesar, and P. B. Godfrey. Finishing Flows Quickly with Preemptive Scheduling. InACM SIGCOMM, 2012

    2012

  74. [74]

    H. D. Dixit, S. Pendharkar, M. Beadon, et al. Silent Data Corruptions at Scale. arXiv:2102.11245, 2021

    Pith/arXiv arXiv 2021

  75. [75]

    must be disabled. . . to prevent silent device memory corruption

    P. H. Hochschild, P. Turner, J. C. Mogul, et al. Cores That Don’t Count. InHotOS, 2021. Stateful Worlds, Stateless Elasticity Appendices A Verification-plane forensics This appendix records the evidence behind the fan-in freeze regime of §6.5. The decision-grade constants and Figure 3 (raw per-chunk timestamps from the 2026-07-04 fabric) stay in the body....

    2021