{"id":"1a6ba15e-0b5b-49cd-ac7a-0cad5ba96d1b","arxiv_id":"2505.22132","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"In a remote rendering testbed over Wi-Fi and 5G, RTP over QUIC achieved lower end-to-end latency than WebRTC, while Media over QUIC had the fastest startup but the highest latency due to its relay.","lead":"This paper measures the latency and connection startup of three video streaming protocols, WebRTC, RTP over QUIC (RoQ), and Media over QUIC (MoQ), in a remote rendering setup over Wi-Fi 6 and a private 5G network. It finds that RoQ was the fastest in both metrics, while MoQ, which routes through a relay, was the slowest in latency.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The QUIC advantage is confounded by implementation maturity: RoQ's Wi-Fi byte loss is 2.67% vs WebRTC's 0.02%, and MoQ's latency is worse than WebRTC, so the 30%/60% claim is not yet a protocol-level result.","rationale":"The reader's weakest assumption, that early-stage GStreamer elements faithfully represent the protocols, is the central risk, and I agree with the conditional verdict. The paper's own tables expose the confound: RoQ's lower latency coincides with materially higher byte loss, while MoQ's worse latency contradicts the abstract's family-level claim. A loss-controlled comparison is needed to separate protocol gains from implementation shortcuts, so the verdict should remain conditional rather than move to acceptance or rejection.","tokens_in":8405,"tokens_out":8752,"duration_ms":92571,"concrete_test":"Run the same Wi-Fi 1080p/15 Mbps setup but insert a network emulator (e.g., Linux netem) with 1% random packet loss between the remote renderer and the player, and measure RoQ and WebRTC end-to-end latency and byte loss over 10 repetitions. If RoQ's latency advantage disappears, or is accompanied by byte loss above 2% while WebRTC remains near 0%, the observed gain is explained by missing error recovery rather than by QUIC protocol design; if RoQ retains the advantage with loss comparable to WebRTC, the maturity concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract: ~30% latency and ~60% startup improvements for QUIC-based protocols) is not established because the measurements conflate protocol design with implementation maturity. Section III-B states RoQ support in GStreamer is \"still in an early stage and lacks full stability\" and that MoQ elements are unofficial and do not support audio. Section IV-A adds that RoQ \"lacks mechanisms for handling transmission errors.\" Table III quantifies this: over Wi-Fi, RoQ has 2.67% byte loss versus 0.02% for WebRTC, and MoQ has 2.85%. If RoQ achieves lower latency in part by not buffering or retransmitting lost packets, the headline latency gain is an artifact of the immature implementation, not an intrinsic QUIC advantage. The internal inconsistency reinforces this: Table I shows MoQ, the other QUIC protocol, has higher latency than WebRTC (e.g., 559.83 ms vs 288.83 ms on Wi-Fi 1080p), so the \"QUIC-based protocols\" generalization is not supported even on this testbed without controlling for implementation maturity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a measurement study of three real-time streaming protocols—WebRTC, RTP over QUIC (RoQ), and Media over QUIC (MoQ)—in the context of a remote-rendered XR/holographic conferencing service. The authors build a Unity-based remote renderer with a GStreamer plugin, deploy it on a Kubernetes cluster with an NVIDIA vGPU, and evaluate the protocols over Wi-Fi 6 and a private 5G SA network. They measure connection startup time, end-to-end latency, CPU/GPU utilization, throughput, jitter, and byte loss at 1080p/720p/480p, and report results in three tables. The central claim, stated in the abstract, is that QUIC-based protocols improve latency by approximately 30% and connection startup by approximately 60% compared to WebRTC.","tokens_in":8655,"tokens_out":4136,"duration_ms":39132,"significance":"If the claims were supported, the paper would be a useful empirical contribution to the ongoing evaluation of QUIC-based media transport for XR services. Its strengths include a realistic private 5G testbed, a distributed containerized deployment, concurrent measurement of resource consumption and network-level metrics, and a direct comparison of two QUIC-based protocols with WebRTC using open-source frameworks. The paper is less persuasive as a protocol-level comparison because the findings are presented as general QUIC advantages while the data show a heterogeneous picture: RoQ often improves latency, MoQ consistently worsens it, and the QUIC implementations are admitted to be immature and loss-prone. The stated 30% latency improvement is therefore not established for 'QUIC-based protocols' as a class.","major_comments":[{"comment":"The abstract's claim that 'Improvements of approximately 30% in latency ... are expected in QUIC-based protocols compared to WebRTC' is contradicted by the paper's own data: MoQ shows higher latency than WebRTC in every configuration (e.g., 559.83 ms vs 288.83 ms on Wi-Fi 1080p), while only RoQ shows an improvement (215.00 ms on the same row). The conclusion in Section V repeats this conflation. The central claim must be restricted to RoQ, or the paper must separately discuss why MoQ's relay-based design incurs higher latency.","section":"Abstract and Section IV-A, Table I"},{"comment":"The latency comparison is confounded by implementation maturity. The authors state that RoQ's GStreamer support 'is still in an early stage and lacks full stability' and that MoQ uses unofficial elements without audio support; Section IV-A adds that RoQ 'lacks mechanisms for handling transmission errors.' Table III quantifies the consequence: RoQ and MoQ exhibit 2.67% and 2.85% byte loss over Wi-Fi versus 0.02% for WebRTC. If RoQ achieves lower latency by omitting retransmission or buffering of lost packets, the reported latency gain cannot be attributed to the QUIC protocol design. The paper should either control for this confound or explicitly frame the results as a comparison of current open-source implementations.","section":"Section III-B and Section IV-A"},{"comment":"All latency, startup, jitter, and loss measurements appear to be single-run values reported without error bars, standard deviations, or repeat counts. For example, Table I reports values such as 948.4 ms and 215.00 ms with no indication of run-to-run variability. Before making percentage-improvement claims, the authors should repeat each experiment multiple times and report dispersion metrics; ideally they should also test whether observed differences are statistically significant given the variability of wireless channels.","section":"Section IV-A, Tables I-III"},{"comment":"The conclusion sentence 'with RoQ achieving the fastest connection startup but suffering from higher overall latency due to its communication mechanism' is internally inconsistent with Table I, which shows RoQ having lower latency than WebRTC in all rows (e.g., 125.67 ms vs 269.92 ms on 5G at 1080p). The 'higher overall latency' description appears to apply to MoQ, not RoQ. The conclusion should be corrected so that it matches the reported data.","section":"Section V"}],"minor_comments":[{"comment":"The phrase 'Gruops of Picture' should be corrected to 'Group of Pictures'.","section":"Section IV-A"},{"comment":"The sentence beginning 'he tests were conducted' should begin with 'The tests were conducted'.","section":"Section III-C"},{"comment":"The explanation that MoQ's relay increases latency is plausible but is not isolated by any measurement; the authors could strengthen it by measuring MoQ in a direct (non-relay) mode or citing a controlled comparison.","section":"Section IV-A"},{"comment":"The column header 'Bytes Loss (%)' should be 'Byte Loss (%)' for consistency with the text and standard terminology.","section":"Table III"},{"comment":"The phrase 'In this research, we propose the development of a solution' describes the implemented system rather than a proposal; rephrasing would align with the experimental nature of the work.","section":"Section II-C"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is not supported by its own measurements: only one of the two QUIC-based protocols improves latency, and the effect is confounded by the acknowledged immaturity of the QUIC implementations. The underlying testbed and measurement scope are valuable, and the paper could be made publishable by substantially revising the claims, adding statistical rigor, and clearly separating protocol-level conclusions from implementation-level observations. I recommend major revision rather than rejection because the data themselves are potentially useful and the issues, while load-bearing, are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the direct three-way comparison. Nobody else has put MoQ, RoQ, and WebRTC in the same remote rendering pipeline over Wi-Fi 6 and private 5G and reported latency, startup, throughput, jitter, and byte loss. The testbed is real: Unity renderer, GStreamer, H.264, hardware encoding, Kubernetes. The resource usage tables are a nice addition. So the paper earns its place as a data point for protocol selection in XR streaming.\n\nThe soft spots are real, though. The abstract claims 'approximately 30% in latency and 60% in connection startup are expected in QUIC-based protocols compared to WebRTC.' That generalization does not hold for MoQ, which is consistently slower than WebRTC in latency (e.g., 559.83 ms vs 288.83 ms on Wi-Fi 1080p). Only RoQ supports the latency claim, and the startup improvement is not 60% in every case. So the headline needs to say 'RoQ' and 'measured', not 'expected'.\n\nMore seriously, the comparison is confounded by implementation maturity. The paper itself says RoQ is 'still in an early stage and lacks full stability' and lacks mechanisms for handling transmission errors, and MoQ elements are unofficial and do not support audio. Table III shows RoQ losing 2.67% of bytes over Wi-Fi vs WebRTC's 0.02%. If RoQ achieves lower latency in part by not buffering or retransmitting, the gain is a property of that implementation, not of QUIC as a protocol. That does not invalidate the measurement—it just means the conclusion should be 'RoQ in this GStreamer implementation' rather than 'QUIC-based protocols'.\n\nOther issues: single-run measurements with no error bars or statistical treatment; the latency extraction method from PCAPs is not described in enough detail to reproduce; software versions are not pinned. The conclusion also contains a confused sentence saying RoQ has 'fastest connection startup but suffering from higher overall latency'—that contradicts Table I and looks like a leftover from an earlier draft where MoQ was meant.\n\nWho is this for? Practitioners evaluating protocol options for remote rendering will find the tables useful, but they should read the body, not the abstract. It deserves peer review because the comparison is new and the testbed is credible, but the authors need to fix the overclaim, add repetitions, and release artifacts.","headline":"A useful first three-way latency comparison of MoQ/RoQ vs WebRTC for remote rendering, but the headline QUIC advantage is overstated and confounded by immature implementations.","tokens_in":9180,"tokens_out":2123,"would_cite":false,"duration_ms":19974,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that RTP over QUIC (RoQ) currently gives the lowest end-to-end latency for remote-rendered XR video over Wi-Fi and 5G, that QUIC-based transports also connect faster than WebRTC, and that Media over QUIC (MoQ) trails on…","keywords":["remote rendering","Extended Reality","QUIC","RTP over QUIC","Media over QUIC","WebRTC","5G","low-latency streaming"],"falsifier":"Repeat the 1080p/30 fps H.264 test over the same private 5G link using a mature, officially maintained RoQ implementation with packet-loss recovery; if end-to-end latency is no longer below WebRTC's, the paper's central claim that RoQ lowers latency is falsified.","tokens_in":8243,"feed_emoji":"⚡","tokens_out":7505,"duration_ms":81309,"temperature":0.7,"pith_summary":"The paper tries to establish which real-time video transport protocol is ready to carry remote-rendered XR content with the least delay. It compares WebRTC, RTP over QUIC (RoQ), and Media over QUIC (MoQ) over Wi-Fi 6 and a private 5G network, streaming H.264 video from a game-engine renderer to a spectator player. Its central result is that QUIC-based transports start connections faster than WebRTC and that RoQ achieves the best end-to-end latency, for example 215 ms versus 288.83 ms for WebRTC at 1080p over Wi-Fi, while MoQ's relay adds roughly double the latency. The paper also finds that WebRTC remains the most mature and stable implementation, with lower jitter and byte loss than the early-stage QUIC stacks.","feed_headline":"Measured remote-render latency: RoQ 215 ms vs WebRTC 289 ms","feed_subtitle":"RTP-over-QUIC beat WebRTC by ~90 ms on Wi-Fi at 1080p and starts sessions faster.","key_machinery":"The load-bearing machinery is the set of three protocol stacks: WebRTC, which negotiates a peer-to-peer channel through a signaling server and then carries RTP; RTP over QUIC (RoQ), which encapsulates RTP packets inside QUIC after a UDP handshake; and Media over QUIC (MoQ), which muxes fragmented MP4 media and pushes it through a relay server that the client subscribes to. The startup differences follow from the negotiation path: QUIC collapses setup into a handshake, while WebRTC needs signaling, and MoQ starts generating traffic toward the relay even before the client redirects it, which explains both its fastest startup and its latency penalty.","core_discovery":"The paper's central claim is that, under the tested conditions, RoQ delivers the lowest end-to-end latency among the three protocols, and both QUIC-based protocols connect faster than WebRTC. The authors attribute the startup advantage to QUIC's single UDP handshake replacing WebRTC's signaling negotiation, and they attribute MoQ's high latency to its relay-based publisher/subscriber delivery path. The abstract's expected figure of roughly 30% latency improvement is supported for RoQ by the measured Table I values, whereas MoQ's measured latency is about 100% higher than WebRTC's. The paper further claims that WebRTC still leads in stability, jitter, and byte-loss behavior, and that the open-source QUIC implementations are not yet mature enough for production use.","pith_inferences":["The abstract's 'approximately 30%' latency improvement is best read as an expected figure for RoQ specifically; MoQ's measured latencies are higher than WebRTC's, so the headline number should not be generalized to all QUIC-based transports.","Part of MoQ's latency penalty may come from fragmented-MP4 muxing and the lack of audio in its early elements, so a follow-up with equivalent encapsulation could separate relay cost from implementation cost.","Because the measurements ran on a private 5G network with no competing traffic, a loaded public network could change the relative ranking; testing under congestion would be the natural extension.","The paper's suggested multipath-QUIC direction could be turned into a concrete test: enable simultaneous 5G and Wi-Fi paths and measure whether latency and startup improve under handover."],"forward_implications":["RoQ becomes the strongest of the three candidates for latency-critical remote rendering of 360-degree video at 30 fps, once its stability approaches WebRTC's.","Connection startup for XR sessions can be shortened substantially by choosing a QUIC-based transport, because MoQ and RoQ both avoid WebRTC's signaling negotiation.","MoQ's relay-based delivery roughly doubles end-to-end latency in these measurements, making it a poor fit for interactive latency-sensitive viewing unless relay overhead is reduced.","The 5G link often yields lower end-to-end latency than Wi-Fi once a session is established, so access-network choice matters after startup even when the protocol choice dominates the initial handshake.","WebRTC remains the robustness benchmark: it had the best jitter and byte-loss behavior, so QUIC adoption needs implementation maturity work rather than latency gains alone."],"supporting_citations":[{"why":"Defines QUIC as the UDP-based multiplexed and secure transport that underlies both QUIC streaming variants tested.","marker":"[12]"},{"why":"RTP over QUIC (RoQ) Internet-Draft that specifies the RoQ encapsulation measured in the paper.","marker":"[14]"},{"why":"Media over QUIC (MoQ) transport draft that specifies the publisher/subscriber relay transport tested as MoQ.","marker":"[15]"},{"why":"Describes WebRTC's real-time RTP-based communication stack, the baseline protocol compared against QUIC.","marker":"[11]"},{"why":"Provides the claimed QUIC benefits including multiplexing, connection migration, and end-to-end encryption that motivate the comparison.","marker":"[6]"},{"why":"QUIC Version 2 specification cited to establish QUIC as a standardized alternative to WebRTC.","marker":"[5]"}],"fun_headline_variants":["RoQ slashes XR latency vs WebRTC, but MoQ lags","RoQ cuts XR latency 25% vs WebRTC, but MoQ is slower","QUIC handshake enables faster XR connection startup than WebRTC","WebRTC still most stable, but RoQ wins XR latency race","MoQ relay path undermines QUIC benefit in XR streaming"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim leans on the assumption that the early-stage, unofficial open-source implementations of RoQ and MoQ faithfully represent the protocols; if those implementations are unrepresentative, the measured latency differences reflect software maturity rather than protocol design.","fun_headline_variants_meta":{"raw":{"variants":["RoQ slashes XR latency vs WebRTC, but MoQ lags","RoQ cuts XR latency 25% vs WebRTC, but MoQ is slower","QUIC handshake enables faster XR connection startup than WebRTC","WebRTC still most stable, but RoQ wins XR latency race","MoQ relay path undermines QUIC benefit in XR streaming"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000791,"raw_usage":{"total_tokens":3461,"prompt_tokens":893,"completion_tokens":2568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":2464}},"tokens_in":509,"tokens_out":2568,"duration_ms":20925,"temperature":1.0,"reasoning_tokens":2464,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:13:51.289881+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the 1080p/30 fps H.264 test over the same private 5G link using a mature, officially maintained RoQ implementation with packet-loss recovery; if end-to-end latency is no longer below WebRTC's, the paper's central claim that RoQ lowers latency is falsified.","supporting_citations":[{"cited_title":"RTP over QUIC (RoQ),","cited_arxiv_id":null,"evidence_quote":"RTP over QUIC (RoQ) Internet-Draft that specifies the RoQ encapsulation measured in the paper."},{"cited_title":"Media over QUIC Transport,","cited_arxiv_id":null,"evidence_quote":"Media over QUIC (MoQ) transport draft that specifies the publisher/subscriber relay transport tested as MoQ."},{"cited_title":"Towards cloud-native virtual reality applications: State-of- the-art and open challenges,","cited_arxiv_id":null,"evidence_quote":"Describes WebRTC's real-time RTP-based communication stack, the baseline protocol compared against QUIC."},{"cited_title":"Quic: Better for what and for whom?","cited_arxiv_id":null,"evidence_quote":"Provides the claimed QUIC benefits including multiplexing, connection migration, and end-to-end encryption that motivate the comparison."},{"cited_title":"QUIC Version 2,","cited_arxiv_id":null,"evidence_quote":"QUIC Version 2 specification cited to establish QUIC as a standardized alternative to WebRTC."}],"review_version":1}