{"id":"66d7d2cd-bb27-491e-83ca-e5e2188f3831","arxiv_id":"2508.11035","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"EMLIO reduces AI training data-loading time and energy by streaming pre-batched TFRecord shards over TCP with out-of-order prefetching, keeping epoch time and energy nearly flat from 0.1 ms to 30 ms RTT.","lead":"EMLIO is a service that moves AI training data in pre-batched chunks from storage to GPU nodes, claiming near-constant speed and energy across network distances. A generalist might read it because it targets the growing energy cost of moving data in large-scale AI training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed I/O-energy result is not supported by the reported measurements: the energy bars are total per-epoch CPU/DRAM/GPU energy, so EMLIO's constant energy simply tracks its constant runtime, not minimized I/O energy.","rationale":"I considered the bandwidth-saturation concern raised by the reader, and it remains a real external-validity limitation: the evaluation uses one GPU node, a 10 GB dataset, and a 10 Gbps link, so the network is never the binding constraint. However, at a fixed bandwidth, a sufficiently windowed multi-stream transfer can remain RTT-agnostic even when bandwidth is lower, so that issue is a limitation on the claimed general scalability rather than an internal flaw in the reported experiments. The energy-attribution issue is more load-bearing because it is internal: the paper's own EnergyMonitor and TimestampLogger make stage-level segmentation possible, yet the headline energy numbers are total epoch energy. EMLIO's near-constant total energy across RTTs is fully explained by its near-constant epoch time at roughly constant average power; the paper never subtracts a no-I/O baseline and never reports energy for the Read+Preprocess stage alone in the method comparisons. If the proposed control shows that DeltaE_EMLIO and DeltaE_DALI are similar, the central energy contribution collapses to a runtime effect, and the claimed novelty of optimizing I/O energy is unsupported. The reader already assigned CONDITIONAL and noted the energy overstatement in the rationale, so my read does not change the verdict; it sharpens the specific test needed before the energy claim can be accepted.","tokens_in":17150,"tokens_out":10563,"duration_ms":115112,"concrete_test":"Run one control epoch of ResNet-50 on the 10 GB ImageNet subset with all samples pre-staged in GPU memory (zero network I/O) and measure total epoch energy E_baseline with EnergyMonitor. Then, at 30 ms RTT, measure total epoch energy E_EMLIO and E_DALI, and compute the I/O-attributable energies DeltaE_EMLIO = E_EMLIO - E_baseline and DeltaE_DALI = E_DALI - E_baseline. If DeltaE_EMLIO is not substantially smaller than DeltaE_DALI, the paper's \"minimal I/O energy\" and 10.9x energy claims are a runtime effect, not an I/O-energy optimization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 (Section 3) and Figures 5-10 measure total CPU-package, DRAM, and GPU energy over the entire epoch via perf stat energy counters and NVML. No baseline with data pre-staged on the compute node is reported, and no stage-level segmentation is applied to the EMLIO-vs-DALI comparisons, even though Figure 1 shows the authors can segment Read, Read+Preprocess, and Train stages. The central assertion that \"I/O-related energy remains minimal\" (Section 5.1.1) is therefore not supported: EMLIO's total epoch energy is near 36 kJ at both local and 30 ms RTT because both epochs take about 156 s at similar average power; the claim is indistinguishable from the trivial statement that a shorter epoch consumes less total energy. The abstract's 10.9x energy reduction is a ratio of total epoch energies against DALI/PyTorch runs that take 11-27x longer, not a measure of energy per byte, per I/O operation, or per Read+Preprocess stage. The Figure 1 caption's note that all reported energies \"contain the ideal energy\" reinforces that no baseline subtraction was performed. Since the paper's stated novelty is joint latency and I/O-energy optimization, the energy pillar of the central claim currently rests on an unsupported attribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents EMLIO, a storage-side data-serving daemon that pre-batches and serializes TFRecord samples, streams them over ZeroMQ/TCP with out-of-order prefetching, and feeds them into a DALI pipeline on compute nodes. The central claims are (i) near-constant per-epoch duration and energy as network RTT grows from 0.1 ms to 30 ms, and (ii) up to 8.6x faster I/O and 10.9x lower energy compared with PyTorch DataLoader and NVIDIA DALI. Evaluation covers ResNet-50 and VGG-19 on ImageNet, COCO, and synthetic 2 MB records in Chameleon cloud, across local, LAN, emulated-LAN, and WAN regimes, with a custom synchronized perf/NVML energy monitor. The latency-hiding results are plausible; the energy-attribution claims are the main weakness.","tokens_in":17374,"tokens_out":4248,"duration_ms":44885,"significance":"If the latency claims hold, EMLIO is a useful contribution to remote and geo-distributed training I/O: the architecture is clean, the results are reported consistently across datasets and models, and the barrier-aligned CPU/DRAM/GPU sampling harness with NTP-synchronized timestamps is a reasonable measurement infrastructure. However, the paper's stated novelty is joint latency and I/O-energy minimization, and the energy pillar currently rests on total-epoch energy measurements with no baseline subtraction or stage-level segmentation. The paper also lacks error bars for its headline comparisons and does not test the bandwidth-constrained regime, so the RTT-invariance claim is only established under abundant bandwidth.","major_comments":[{"comment":"The claim that 'I/O-related energy remains minimal' is not supported by the reported measurements. Figures 5–10 report total per-epoch CPU/DRAM/GPU energy over the entire training epoch, and the Figure 1 caption explicitly states that all reported energies 'also contain the ideal energy.' EMLIO's near-constant energy at 30 ms RTT (about 36 kJ) therefore simply tracks its near-constant runtime (about 156 s), while DALI and PyTorch consume more total energy because their epochs run 3.5–27x longer. No baseline with data pre-staged on the compute node is reported, and no stage-level segmentation is applied in Figures 5–10 even though Figure 1 demonstrates the ability to segment Read, Read+Preprocess, and Train stages. Consequently, the abstract's 10.9x lower-energy claim is a ratio of total epoch energies, not a reduction in I/O energy per byte, per I/O operation, or per Read+Preprocess stage. The authors should either (a) report energy per stage with the ideal training energy subtracted, or (b) explicitly reframe all energy claims as end-to-end epoch energy and remove the I/O-energy attribution.","section":"§5.1.1, Fig. 5; Fig. 1 caption"},{"comment":"The headline numbers are internally inconsistent. The abstract states 'up to 8.6X faster I/O and 10.9X lower energy use,' while the conclusion states EMLIO 'cuts I/O energy by up to 8× and speeds up epochs by up to 13×' over the same baselines. Figure 5 at 30 ms RTT shows EMLIO is about 10.9x faster than DALI and about 27x faster than PyTorch, and the energy ratios there are also different from both the abstract and the conclusion. The paper should state exactly which configuration (dataset, model, RTT, baseline) produces each headline number, and the abstract and conclusion should be reconciled.","section":"Abstract vs. §6"},{"comment":"The RTT-invariance result is established only in a bandwidth-abundant regime. All experiments use a 10 Gbps link with a 10 GB dataset on a single GPU node; the required data rate is roughly 10 GB / 156 s ≈ 64 MB/s, far below the 10 Gbps link capacity. In this regime, latency hiding via prefetching and multi-streaming is expected to work, and the result does not test the case where bandwidth, not round-trip time, is the binding constraint. The claim that EMLIO 'fully masks network delays' should be tested with a higher per-GPU data rate (e.g., larger datasets, multiple concurrent GPUs, or a lower-bandwidth link) to show that the constant-duration result is not an artifact of underutilized network capacity.","section":"§4.1, §5.1.1"},{"comment":"No error bars, confidence intervals, or run counts are reported for the central energy and duration comparisons. Figure 11 shows measurable run-to-run variability (±1 standard deviation over three runs), which suggests the headline comparisons in Figures 5–10 should include at least mean and standard deviation over multiple runs. Without this, the quantitative claims (e.g., 'less than 5% variation' in epoch time) cannot be distinguished from noise.","section":"Figs. 5–10"}],"minor_comments":[{"comment":"The keyword 'energy-efficency' is misspelled; it should be 'energy-efficiency.'","section":"Keywords"},{"comment":"The caption refers to 'Section III,' but the paper uses numeric section numbering; it should refer to 'Section 3.'","section":"Fig. 1 caption"},{"comment":"The text says 'Initialize NVML, TMDS client'; 'TMDS' appears to be a typo and should likely read 'TSDB/InfluxDB client.'","section":"Algorithm 1 explanation"},{"comment":"The GPU energy formula contains a corrupted summation symbol (shown as '˝') and should be typeset as ∑ᵢ Pᵢ · δ / 1000.","section":"Algorithm 1, line 11"},{"comment":"The two panels of Figure 8 use different y-axis scales (0.1 ms and 1 ms), which makes visual comparison difficult; consider adding shared or annotated scales.","section":"Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The I/O-energy claim is the paper's main differentiator from prior data-loading systems. If the authors cannot provide stage-separated or baseline-subtracted energy measurements, the paper should be reframed as a latency-hiding system with end-to-end energy co-benefits; in its current form the energy contribution is overstated. The bandwidth-sufficiency issue also needs to be addressed before the RTT-invariance claim can be considered general."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: EMLIO's latency results are worth knowing; its energy claims, as written, do not survive contact with the measurements. The paper deserves a serious referee, but with a demand to re-frame the energy story.\n\nWhat is actually new: EMLIO offloads batch construction to the storage side—mmap TFRecord shards, serialize fixed-size batches, push over parallel ZeroMQ streams with out-of-order prefetching and backpressure, then hand msgpack payloads to DALI's external_source on the compute node. The pieces are known (NoPFS-style prefetching, TFRecord sharding, DALI integration), but the storage-side pre-batching daemon and the explicit time-plus-energy objective are a fair synthesis. The evaluation is the strong part: three datasets, two models, four RTT regimes, and the central result is credible—EMLIO's epoch time stays near 156 s from local disk out to 30 ms RTT, while DALI and PyTorch degrade to 1.7–4.2k s. On a single GPU node with 10 Gbps, network distance becomes invisible. That is a useful result.\n\nSoft spots, in order of size. The I/O-energy claim is an attribution error. EnergyMonitor measures total CPU/DRAM/GPU energy over the whole epoch (Algorithm 1, Figures 5–10), and Figure 1's caption admits the numbers \"contain the ideal energy\"—no idle baseline is subtracted. EMLIO's flat ~36 kJ per epoch is its flat ~156 s epoch restated in joules. The sentence \"I/O-related energy remains minimal\" (Section 5.1.1) never isolates the Read+Preprocess stage, and the abstract's 10.9x reduction is a ratio of total epoch energies against runs that take 11–27x longer. That may be a real energy-bill saving, but it is not demonstrated I/O-energy minimization. The fix is inside the paper: Figure 1 shows they can segment stages, so a pre-staged local baseline plus stage-level subtraction would support or retire the claim.\n\nSecond, the paper's own Scenario 2 undercuts the constant-energy story: with half-local/half-remote shards, EMLIO's runtime holds at ~222 s across RTTs while CPU energy climbs from 19.7 to 52.5 to 106 kJ. That is not an energy profile that stays constant, and it suggests the energy measurements are noisier than the text admits. Smaller nits: abstract says 8.6x/10.9x while the conclusion says 13x/8x; Section 5.3's 7500 s DALI COCO epoch is out of line with Figure 6's ~1.3k s; no error bars on the main comparisons; no code or data released; and \"linear horizontal scalability\" is asserted but never tested—Scenario 2 is one node, not a cluster.\n\nWho this is for: anyone building data loaders for geo-distributed training will want the RTT-masking design; the energy section is a good referee exercise in measurement honesty. I would send it to peer review and lean on revision for the energy attribution.","headline":"EMLIO's RTT-immune data-loading design is credible and clearly evaluated, but the headline I/O-energy claim is total-epoch energy mislabeled as I/O energy, and the paper needs honest re-framing before its energy title is earned.","tokens_in":17929,"tokens_out":9340,"would_cite":true,"duration_ms":95168,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes EMLIO, a service-based I/O design that keeps data-loading latency and energy nearly constant across network distances by moving batch preparation into a storage-side daemon.","keywords":["I/O latency","energy efficiency","deep learning","data loading","distributed storage","GPU-accelerated preprocessing","TFRecord","out-of-order prefetching"],"falsifier":"A single test at 1 Gbps with the same 10 GB ImageNet subset would settle it: if per-epoch time grows with RTT or exceeds the local-disk epoch by more than five percent, the claim that latency alone is masked fails.","tokens_in":16933,"feed_emoji":"⚡","tokens_out":7323,"duration_ms":69759,"temperature":0.7,"pith_summary":"This paper argues that the I/O stage of distributed AI training can be made nearly insensitive to network distance if the storage side, not the training process, does the batching and serialization. EMLIO places a lightweight daemon on storage servers that reads raw samples, packs them into pre-sized batches, and streams those batches over parallel TCP channels, while a client-side receiver feeds them into GPU-accelerated preprocessing. Across local disk, LAN, and WAN conditions, the authors report per-epoch time and I/O energy varying by less than five percent from 0.1 ms to 30 ms round-trip time, and up to 8.6x faster I/O and 10.9x lower energy than existing loaders. The reason to care is that geo-distributed and cloud training currently idle GPUs and waste energy waiting on remote reads; if this holds, distance stops being the dominant cost.","feed_headline":"Pre-batched streaming hides network delay in AI training I/O","feed_subtitle":"EMLIO keeps epoch time and I/O energy nearly flat from 0.1 ms to 30 ms RTT, beating standard loaders.","key_machinery":"The load-bearing mechanism is the storage-side EMLIO Daemon, which memory-maps large TFRecord shards, serializes fixed-size batches into msgpack payloads, and pushes them over multiple ZeroMQ/TCP streams with high-water-mark backpressure; the compute side runs an EMLIO Receiver that deserializes into a shared queue and feeds DALI's external_source, so GPU decoding and augmentation overlap with network transfer. Out-of-order prefetching across parallel send threads lets the pipeline exploit storage and network parallelism. The secondary machinery is the distributed EnergyMonitor: barrier-synchronized CPU/DRAM and GPU samplers at 100 ms intervals write NTP-aligned energy tuples to a time-series database, letting the authors attribute energy per epoch.","core_discovery":"The central discovery is that a service-based I/O design, an EMLIO Daemon co-located with storage that memory-maps TFRecord shards, serializes groups of samples into msgpack batches, and pushes them over ZeroMQ/TCP streams with backpressure, can hide round-trip latency almost entirely. A synchronized energy monitor built from perf and NVML sampling shows the energy footprint stays flat as network distance grows. In evaluation, EMLIO's ImageNet epoch time stays near 156 s across 0.1 ms, 10 ms, and 30 ms RTT, where the standard PyTorch loader and DALI slow by 3x to 27x and consume 4x to 60x more energy. The same pattern holds for COCO and for VGG-19, and in a sharded local-plus-remote setup EMLIO reduces epoch time by up to 18.7x at 30 ms RTT. To the authors' knowledge, this is the first end-to-end measurement of I/O energy during model training and the first loader designed to minimize both latency and energy.","pith_inferences":["The RTT-agnostic result is demonstrated only where bandwidth (10 Gbps) exceeds GPU consumption; a natural extension is to test at 1 Gbps or with larger samples, where the daemon's serialization and stream count, not latency, should become the bottleneck.","Because EMLIO amortizes per-batch setup across many samples, the optimal batch size and daemon concurrency likely need to grow with record size; the paper shows concurrency 2 recovering the lead for 2 MB records, implying a tunable knob for other record distributions.","A testable extension, which the authors themselves flag, is to combine EMLIO with gradient-synchronization scheduling so that data arrival and collective communication are co-optimized rather than treated as separate stages.","If the energy numbers are validated by independent measurement, the same synchronized-sampling methodology could become a standard energy benchmark for data loaders, not just for EMLIO."],"forward_implications":["If the central claim is right, geo-distributed AI training can place datasets on cheap remote storage without paying an epoch-time or energy penalty, as long as bandwidth is not the binding constraint.","Energy-aware provisioning becomes possible: because I/O energy is roughly constant across RTT, data-movement energy can be treated as a fixed per-epoch cost rather than a distance-dependent one.","A horizontal scaling path follows from the design: adding storage nodes with their own daemons increases aggregate throughput linearly, so the mechanism transfers to multi-node dataset sharding.","The pipeline should keep paying off as accelerators get faster, because the bottleneck is shifted from remote read latency to pre-batching and streaming concurrency on the storage side.","Wall-clock convergence improves at high latency: the paper shows ResNet-50 loss reaching the same level in about 1,000 s under 10 ms RTT where DALI takes 7,500 s."],"supporting_citations":[{"why":"Supplies the out-of-order prefetching idea that EMLIO adapts to hide network latency.","marker":"[11]"},{"why":"ResNet-50 is the workload model for most epoch-time and energy measurements.","marker":"[16]"},{"why":"DALI is the GPU-accelerated preprocessing library EMLIO plugs into and the main comparison baseline.","marker":"[33]"},{"why":"TFRecord is the sharded file format whose mmap-friendly layout EMLIO's batching relies on.","marker":"[48]"},{"why":"ImageNet is the standard dataset used in the core RTT-versus-epoch-time experiment.","marker":"[10]"},{"why":"Provides the CPU package and DRAM energy readings used by the EnergyMonitor.","marker":"[1]"},{"why":"Provides the GPU power readings used by the EnergyMonitor.","marker":"[35]"},{"why":"Stores the NTP-aligned energy time series that make per-epoch attribution possible.","marker":"[17]"},{"why":"Supplies the cloud testbed nodes, networking, and 10 Gbps fabric used in all experiments.","marker":"[22]"}],"fun_headline_variants":["EMLIO flattens AI training I/O latency and energy across networks","Service-based I/O hides network delay, cuts energy 10.9x in AI training","Energy-aware I/O keeps AI training fast even at 30ms RTT","EMLIO: 8.6x faster I/O, 10.9x less energy for AI training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result assumes the network pipe is wide enough to carry pre-batched data faster than the GPU consumes it; only round-trip latency is varied, so the claim is not established when bandwidth is the binding constraint.","fun_headline_variants_meta":{"raw":{"variants":["EMLIO flattens AI training I/O latency and energy across networks","Service-based I/O hides network delay, cuts energy 10.9x in AI training","Energy-aware I/O keeps AI training fast even at 30ms RTT","EMLIO: 8.6x faster I/O, 10.9x less energy for AI training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3311,"prompt_tokens":997,"completion_tokens":2314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":2220}},"tokens_in":613,"tokens_out":2314,"duration_ms":18275,"temperature":1.0,"reasoning_tokens":2220,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:28:37.381520+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A single test at 1 Gbps with the same 10 GB ImageNet subset would settle it: if per-epoch time grows with RTT or exceeds the local-disk epoch by more than five percent, the claim that latency alone is masked fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ResNet-50 is the workload model for most epoch-time and energy measurements."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DALI is the GPU-accelerated preprocessing library EMLIO plugs into and the main comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ImageNet is the standard dataset used in the core RTT-versus-epoch-time experiment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CPU package and DRAM energy readings used by the EnergyMonitor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the GPU power readings used by the EnergyMonitor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Stores the NTP-aligned energy time series that make per-epoch attribution possible."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the cloud testbed nodes, networking, and 10 Gbps fabric used in all experiments."}],"review_version":2}