{"id":"3b986e4b-1861-4097-a383-16662c0f3748","arxiv_id":"2507.00623","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A VR remote-rendering testbed measures end-to-end latency over Wi-Fi and 5G for WebRTC, DASH, LL-DASH, QUIC, and MoQ, with Unity's Render Streaming lowest at 82 ms.","lead":"This paper builds a remote-rendering testbed for virtual reality, offloading VR graphics to a GPU server and streaming 360-degree video to lightweight devices over WebRTC, DASH, QUIC, and MoQ. It measures and compares end-to-end latency over Wi-Fi and 5G, plus maximum frame rates for GStreamer and FFmpeg integrations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Protocol comparisons in Tables I/II are confounded: each protocol row also changes the sender and/or receiver implementation, so the reported ordering cannot be attributed to the protocol itself.","rationale":"The reader's weakest assumption is NTP/measurement accuracy. That concern is real and underdocumented, but a constant clock bias would shift all rows roughly equally and is therefore less threatening to relative claims than the design confound. The paper is transparent about implementation differences and even labels sender/receiver in each row, but its conclusions (e.g., 'MoQ implementation is already outperforming GStreamer WebRTC') promote these confounded measurements to protocol-level findings. A controlled ablation is the minimal check. I still credit the testbed construction, the explicit caveats about non-compliant GStreamer DASH and unofficial MoQ, and the concrete latency tables; the issue is not fabrication but interpretability. The conditional verdict stands.","tokens_in":8479,"tokens_out":8067,"duration_ms":101249,"concrete_test":"Re-run the latency measurement as a controlled ablation: feed one encoded H.264 stream from the same Unity+GStreamer sender into WebRTC, MoQ, and QUIC-RTP sinks, and play all three with the same receiver code (e.g., one GStreamer player with interchangeable protocol elements). If the relative ordering of URS WebRTC, MoQ, GStreamer WebRTC, and QUIC-RTP changes or collapses, the current protocol-level comparisons in Table I are confounded. For DASH, compare GStreamer and FFmpeg with identical segmenter settings and a common Dash.js version, reporting segment and fragment durations plus trial counts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim that the measured latencies characterize WebRTC, DASH, LL-DASH, QUIC, and MoQ, each protocol comparison must hold the surrounding pipeline fixed. Tables I and II do not: URS WebRTC uses Unity's URS sender and player; GStreamer WebRTC uses a gstwebrtc-api receiver; DASH rows use GStreamer or FFmpeg senders with Dash.js; QUIC-RTP uses a custom GStreamer receiver; MoQ uses an unofficial MoQ player. Thus the 29-77 ms gaps between URS WebRTC, MoQ, GStreamer WebRTC, and QUIC-RTP, and the +156% GStreamer-DASH vs FFmpeg-DASH gap, are attributable to encoder insertion, muxing, player buffering, or implementation maturity as much as to the protocol. The paper even attributes WebRTC differences to GPU frame extraction rather than WebRTC. Accurate NTP would improve precision but would not remove this confound; a constant clock offset would not make the protocol labels valid. This is the load-bearing weakness in the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a remote-rendering testbed that integrates Unity with three multimedia frameworks (Unity Render Streaming, GStreamer, and FFmpeg) and delivers a rendered 360-degree VR scene over WebRTC, DASH, LL-DASH, QUIC-RTP, and MoQ. The authors measure the maximum rendering throughput and the end-to-end latency over Wi-Fi and a 5G testbed, reporting in Tables I and II average latencies from 82 ms (URS WebRTC) to 9399 ms (GStreamer DASH) at 1080p/60fps/10 Mbps. The paper concludes that URS WebRTC has the lowest latency, that MoQ and QUIC are promising near-term alternatives, and that DASH/LL-DASH have the highest latency but offer scalability. The main claimed contributions are a multi-protocol remote-rendering testbed and a comparative performance analysis of frameworks and protocols.","tokens_in":8613,"tokens_out":6646,"duration_ms":68500,"significance":"If the measurements were properly controlled, the testbed would be a useful community resource for XR remote-rendering research, and the comparison of five delivery protocols over both Wi-Fi and a 5G testbed is a valuable data point. The paper is honest in reporting implementation details and in attributing some gaps to framework behavior rather than to protocol properties; the authors also flag a GStreamer DASH compliance issue and submit a merge request, which is a positive community contribution. However, the current presentation does not separate protocol effects from implementation effects, and the latency methodology is too underspecified for the numerical ordering to be taken as a protocol comparison. The engineering contribution is real, but the central quantitative claim needs substantial revision and additional measurement detail before it can be accepted.","major_comments":[{"comment":"The central claim that the measured latencies characterize the protocols is undermined by the fact that each protocol row changes the sender framework, the receiver implementation, or both; for example, URS WebRTC uses the URS sender and the URS player, GStreamer WebRTC uses a gstwebrtc-api receiver, the DASH rows use different senders (GStreamer versus FFmpeg) with the same Dash.js receiver, QUIC uses a custom GStreamer pipeline as receiver, and MoQ uses an unofficial MoQ player. The latency gaps in Tables I and II (82 vs 219 ms for WebRTC, 6183 vs 9399 ms for DASH) are therefore attributable to encoder integration, muxing, player buffering, or implementation maturity rather than to the protocol itself, as the paper itself acknowledges when explaining the WebRTC gap by GPU frame extraction. To support the protocol-level conclusion, the authors need either controlled experiments that vary only the protocol while holding the surrounding pipeline fixed, or a reframing of the results as comparisons of complete framework-protocol stacks rather than of protocols.","section":"Section IV, Tables I and II"},{"comment":"The latency measurement methodology is under-specified; the paper does not report the number of repetitions, the statistical meaning of Latencydev (standard deviation, range, or other), the procedure for extracting the frame timestamp from the rendered clock object, the matching of received frames to rendered timestamps, or the NTP synchronization accuracy and residual clock offset. Without these details, the reader cannot assess whether the reported differences (e.g., 29 ms between QUIC-RTP and GStreamer WebRTC over Wi-Fi) are significant, and a constant offset from imperfect clock synchronization would shift every value in the tables. The authors should describe the measurement procedure precisely and provide confidence intervals or per-repetition distributions.","section":"Section IV, Figure 4 and Tables I and II"},{"comment":"The reported DASH and LL-DASH latencies (6183 ms and 4873 ms over Wi-Fi) are not reconciled with the configured segment duration of 2 seconds and fragment duration of 0.5 seconds; the paper does not explain what components contribute the remaining latency (encoder GOP, muxing, buffer, player, etc.) or why LL-DASH with 0.5 second fragments still yields about 4.9 seconds of end-to-end latency. Without a latency budget breakdown, the DASH/LL-DASH comparison is not interpretable, and the claim that these values characterize the protocols is not supported.","section":"Section IV, Table I"}],"minor_comments":[{"comment":"Reference [24] duplicates reference [2] (Yeregui et al., IEEE BMSB 2024); please remove the duplicate or cite different work.","section":"References"},{"comment":"The GStreamer merge request URL contains a space ('merge requests/7886'); it should likely be 'merge-requests/7886' with a hyphen.","section":"Section IV"},{"comment":"The statement that 'all the protocols show better results over WIFI than 5G, ranging from -36% with URS WebRTC to -4% with LL-DASH' should specify the sign convention and the base of the percentage (e.g., relative reduction of the 5G latency when using Wi-Fi).","section":"Section IV"},{"comment":"The abstract and introduction mention MoQ as a 'protocol' comparable to WebRTC and DASH, but MoQ is an application-layer protocol that runs over QUIC; a sentence clarifying the protocol-stack layer would avoid confusion.","section":"Abstract and Introduction"},{"comment":"Figure 5 is introduced without a clear description of the measurement procedure (e.g., how many seconds are averaged, how the mean is computed, and whether error bars are shown); please add this information or refer to it in the text.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a networking/multimedia venue, and the engineering is real; the main obstacle is that the title and abstract promise a protocol comparison that the data cannot support as presented. The authors should be encouraged to either control for the implementation confounds (for example, by using the same encoder/muxer/player across protocols where possible and by reporting a latency budget) or to explicitly limit the claims to the particular stacks tested. I would not reject the paper, but the revision needs to be substantive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Readable engineering paper and a genuinely useful testbed. The authors integrated GStreamer and FFmpeg with Unity Render Streaming and report end-to-end latency for WebRTC, DASH, LL-DASH, QUIC-RTP, and MoQ over both Wi-Fi and a 3GPP Release 15 5G testbed. The QUIC and MoQ numbers are new, and the testbed itself is a contribution for the XR streaming community. I also credit the transparency: they openly flag the GStreamer DASH non-compliance, the unofficial MoQ implementation, and the fact that QUIC here is only raw RTP over QUIC with no application layer.\n\nThe main soft spot is the one the stress test identifies: Tables I and II are presented as a protocol comparison, but each row changes the sender and/or receiver implementation. URS WebRTC uses Unity's sender and player; GStreamer WebRTC uses a different receiver; DASH rows use GStreamer or FFmpeg senders with Dash.js; QUIC uses a custom receiver; MoQ uses an unofficial player. So the 29–77 ms gaps and the huge DASH differences are attributable to encoder insertion, muxing, player buffering, and implementation maturity as much as to the protocol itself. The paper even acknowledges this for WebRTC, attributing the URS-vs-GStreamer gap to GPU frame extraction, but the abstract and conclusions still frame the results as a protocol analysis. That is a genuine load-bearing weakness, not a minor quibble.\n\nSeparately, the measurement methodology is under-specified: no trial count, no confidence intervals, no NTP offset error analysis, and no description of how the rendered timestamp was aligned with the received frame in the player. The numbers are plausible and internally consistent, but we cannot assess their precision. This is fixable with a methods paragraph and ideally by releasing the testbed code and raw data.\n\nWho is this for? Practitioners building cloud/edge VR streaming systems, and researchers benchmarking emerging QUIC-based media transport. It deserves a serious referee: the engineering integration is real, the measurement domain is relevant, and the limitations are mostly correctable. I would accept it for peer review, but the revision needs to reframe the tables as framework-and-protocol combinations rather than pure protocol comparisons, and add the missing methodological detail.\n\nRecommendation: send to review, conditional on fixing the framing and the measurement reporting.","headline":"A useful engineering testbed with novel QUIC/MoQ latency numbers, but the protocol-comparison framing overreaches because rows are confounded with framework and player changes.","tokens_in":9240,"tokens_out":1530,"would_cite":true,"duration_ms":18665,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A remote-rendering testbed delivers VR video with 82 ms latency over WebRTC, 159 ms over MoQ, and 248 ms over RTP-over-QUIC on Wi-Fi, while DASH-based streaming lags by seconds.","keywords":["remote rendering","virtual reality","WebRTC","DASH","LL-DASH","QUIC","Media over QUIC","GStreamer"],"falsifier":"Redo the Wi-Fi and 5G latency measurements with both endpoints synchronized to a shared hardware clock, such as a PTP grandmaster, and compare the resulting values; if the WebRTC latency shifts by more than the reported standard deviation of 12 ms, the NTP-based absolute figures are not reliable and the ranking would need re-examination.","tokens_in":8234,"feed_emoji":"🥽","tokens_out":6235,"duration_ms":68936,"temperature":0.7,"pith_summary":"The paper's central claim is that a complete remote-rendering pipeline for virtual reality can be built from a rendering engine plus two general-purpose media frameworks, and that this pipeline can stream the rendered 360-degree view to lightweight clients over WebRTC, DASH, LL-DASH, QUIC, and Media over QUIC (MoQ). The authors report end-to-end latencies measured over both Wi-Fi and a 5G testbed, with WebRTC via the engine's native plugin having the lowest latency (82 ms average on Wi-Fi), followed by MoQ (159 ms) and QUIC-transported RTP (248 ms), while DASH-based delivery lands in the multi-second range. The contribution is not just a set of numbers: it is an integrated testbed that exposes the practical cost of taking rendered frames out of the GPU, pushing them through a framework pipeline, and packaging them with each protocol. If the measurements hold, a developer choosing a delivery protocol for cloud-rendered VR is trading interactivity against scalability in the way the protocol designs suggest, but with concrete implementation overheads that vary by framework.","feed_headline":"VR remote rendering hits 82 ms via WebRTC, 159 ms via MoQ","feed_subtitle":"A seven-protocol testbed ranks streaming options for offloaded VR graphics over Wi-Fi and 5G.","key_machinery":"The load-bearing object is the bridge between the Unity renderer and the media frameworks: a native C++ plugin that feeds raw audio and 360-degree video buffers into GStreamer or FFmpeg pipelines, with the Unity Render Streaming plugin handling the native WebRTC path. Each protocol is realized as a different pipeline end: GStreamer produces WebRTC, DASH, RTP-over-QUIC, and MoQ (fMP4 over QUIC), while FFmpeg produces DASH and LL-DASH. The latency measurement mechanism is a clock object rendered inside the scene: the rendered frame carries a timestamp visible to the player, and both machines synchronize to a public NTP server, so the difference between generation and display becomes the reported end-to-end latency.","core_discovery":"On the paper's own terms, the discovery is that remote rendering with standard streaming protocols is ready to serve as a research platform: a Unity-based renderer with a GPU encoder can output H.264 1080p/60fps video and, depending on the integration path, deliver it with average latencies from 82 ms (native WebRTC plugin) to roughly 9.4 seconds (GStreamer DASH) over Wi-Fi, and from 129 ms to 10.2 seconds over 5G. The ordering is consistent across both networks, and the paper attributes the gaps among WebRTC variants to the cost of copying raw frames out of the GPU and back in for encoding. It reports that the QUIC-based transport option, RTP over QUIC, lands within 29–40 ms of the GStreamer WebRTC implementation, and that the unofficial MoQ implementation already beats that WebRTC path. DASH and LL-DASH provide scalability but at the cost of seconds of latency, with LL-DASH's fragmented MP4 reducing latency by roughly one fifth relative to plain DASH.","pith_inferences":["A natural next test is measuring the interaction channel separately: the paper's latency numbers cover video delivery, but end-to-end interactivity also includes the uplink of headset motion, which only the URS WebRTC path currently supports.","Because the QUIC option is raw RTP over QUIC without an application layer, its 248 ms Wi-Fi result likely underestimates what RoQ or MoQ could achieve once receiver buffering and congestion control mature; the paper's own MoQ result suggests the gap is already closing.","The same testbed could be extended to variable bitrate and loss conditions, since all tests appear to run at a fixed 10 Mbps; the protocol ranking may change when congestion control and adaptive bitrate are exercised.","If the NTP synchronization error is a few tens of milliseconds, the absolute latency values are less important than the relative ordering, but sub-100 ms claims like 82 ms should be re-validated under hardware-synchronized clocks."],"forward_implications":["Interactive cloud-rendered VR is feasible over WebRTC at the sub-100 ms level on Wi-Fi, since the native WebRTC path averaged 82 ms and the GStreamer path 219 ms.","MoQ, despite being an unofficial early implementation, is the closest QUIC-based protocol to WebRTC latency, supporting the possibility that QUIC-based application protocols can be competitive for immersive streaming.","DASH and LL-DASH cannot support tight interaction loops in this setup; their multi-second latencies restrict them to rotation-only 360 video or non-interactive viewing.","The framework choice matters as much as the protocol choice: copying frames through native APIs is faster than piping through a CLI, and the GStreamer DASH output needed patching for player compatibility.","The 5G results preserve the protocol ordering but add roughly 36–56 ms to the best WebRTC path and smaller relative increases elsewhere, showing that the delivery network is not the dominant cost in this testbed."],"supporting_citations":[{"why":"Provides the Unity Render Streaming plugin that supplies the native WebRTC sender and receiver, anchoring the lowest-latency baseline.","marker":"[25]"},{"why":"Supplies the GStreamer WebRTC receiver used for the comparison WebRTC path.","marker":"[34]"},{"why":"Dash.js is the reference web player used for both DASH and LL-DASH latency measurements.","marker":"[35]"},{"why":"Provides the unofficial Media over QUIC GStreamer element that produces the MoQ measurements.","marker":"[30]"},{"why":"Defines the QUIC transport used by the RTP-over-QUIC and MoQ pipelines.","marker":"[8]"},{"why":"Defines the RTP packaging carried over QUIC in the QUIC latency measurement.","marker":"[29]"},{"why":"GStreamer is one of the two frameworks whose native APIs are integrated via the plugin, producing WebRTC, DASH, QUIC-RTP, and MoQ pipelines.","marker":"[14]"},{"why":"FFmpeg is the CLI-based framework producing the DASH and LL-DASH pipelines that anchor the high-latency end of the comparison.","marker":"[15]"}],"fun_headline_variants":["Remote VR rendering: WebRTC 82 ms, DASH 9.4 s, MoQ on top","VR remote rendering: WebRTC fastest at 82 ms, DASH at 9.4 s","Remote VR streaming: WebRTC hits 82 ms, DASH takes seconds","WebRTC beats DASH for remote VR: 82 ms vs 9.4 s","Remote VR testbed: WebRTC 82 ms, MoQ even faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that the NTP-synchronized clock rendered into the scene gives an accurate timestamp of when each frame was produced, and that the player's clock is aligned closely enough that the measured delays are not systematically offset by synchronization error.","fun_headline_variants_meta":{"raw":{"variants":["Remote VR rendering: WebRTC 82 ms, DASH 9.4 s, MoQ on top","VR remote rendering: WebRTC fastest at 82 ms, DASH at 9.4 s","Remote VR streaming: WebRTC hits 82 ms, DASH takes seconds","WebRTC beats DASH for remote VR: 82 ms vs 9.4 s","Remote VR testbed: WebRTC 82 ms, MoQ even faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001467,"raw_usage":{"total_tokens":5905,"prompt_tokens":955,"completion_tokens":4950,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":4829}},"tokens_in":571,"tokens_out":4950,"duration_ms":35317,"temperature":1.0,"reasoning_tokens":4829,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:10:21.579394+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Redo the Wi-Fi and 5G latency measurements with both endpoints synchronized to a shared hardware clock, such as a PTP grandmaster, and compare the resulting values; if the WebRTC latency shifts by more than the reported standard deviation of 12 ms, the NTP-based absolute figures are not reliable and the ranking would need re-examination.","supporting_citations":[{"cited_title":"Unity Renderer Streaming,","cited_arxiv_id":null,"evidence_quote":"Provides the Unity Render Streaming plugin that supplies the native WebRTC sender and receiver, anchoring the lowest-latency baseline."},{"cited_title":"gstwebrtc-api,","cited_arxiv_id":null,"evidence_quote":"Supplies the GStreamer WebRTC receiver used for the comparison WebRTC path."},{"cited_title":"Dash.js,","cited_arxiv_id":null,"evidence_quote":"Dash.js is the reference web player used for both DASH and LL-DASH latency measurements."},{"cited_title":"Media over QUIC,","cited_arxiv_id":null,"evidence_quote":"Provides the unofficial Media over QUIC GStreamer element that produces the MoQ measurements."},{"cited_title":"QUIC: A UDP-Based Multiplexed and Secure Transport,","cited_arxiv_id":null,"evidence_quote":"Defines the QUIC transport used by the RTP-over-QUIC and MoQ pipelines."},{"cited_title":"RTP: A Transport Protocol for Real-Time Applications,","cited_arxiv_id":null,"evidence_quote":"Defines the RTP packaging carried over QUIC in the QUIC latency measurement."},{"cited_title":"GStreamer,","cited_arxiv_id":null,"evidence_quote":"GStreamer is one of the two frameworks whose native APIs are integrated via the plugin, producing WebRTC, DASH, QUIC-RTP, and MoQ pipelines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FFmpeg is the CLI-based framework producing the DASH and LL-DASH pipelines that anchor the high-latency end of the comparison."}],"review_version":1}