{"id":"b628fbb3-915d-487c-ad88-80fc03db3f9e","arxiv_id":"2505.13169","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":11,"one_line_summary":"RIFLES schedules federated learning clients by forecasting device availability with a CNN-LSTM model, claiming faster convergence and lower dropout than Random, FedCS, and REFL in simulation.","lead":"RIFLES is a scheduling system for federated learning that predicts when phones and other devices are free to train and then picks the best devices for each round. It reports faster and more accurate joint model training, with 10-50% gains over three older methods in simulated mobile settings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 10–50% improvements rest on a synthetic availability generator that also trains the forecaster; without a real-trace or distribution-shift test, the central empirical claim is unsecured.","rationale":"The reader's weakest_assumption is the same one I would flag: the synthetic availability generator is doing double duty as training data and as the online environment, so the paper's headline numbers only demonstrate that RIFLES can exploit the exact periodicity it was trained on. This is load-bearing because the entire method is a forecast-then-schedule loop: if the CNN-LSTM's forecasts degrade under distribution shift, the eligibility matrix and both scheduling policies degrade with it. The absence of an oracle/no-forecast ablation makes it impossible to tell whether forecasting adds anything over the eligibility constraint. I also share the reader's view that the NP-hardness proof in Lemma 2 is unsound (setting alpha=beta=100% removes the capacity constraints, and the swap step is not a valid reduction), which further supports rejection, although that flaw is secondary to the empirical concern. Since the reader already reached REJECT and my analysis does not change that conclusion, the verdict remains UNCHANGED.","tokens_in":12683,"tokens_out":8777,"duration_ms":93754,"concrete_test":"Re-run the WISDM and CIFAR-10 experiments with the identical RIFLES code but replace the synthetic availability generator in Section VII-A2 with a public real-world availability trace, e.g., the PhoneLab smartphone trace or the device availability log used in FLASH. If RIFLES-GH/LRU no longer beat REFL by the claimed 10-50% in accuracy/test loss, or if their completion-rate advantage reverses, the headline result is an artifact of the synthetic generator rather than of availability forecasting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that the availability generator in Section VII-A2 is a faithful proxy for real clients. It instantiates a 1.5x nighttime boost, a 20% hourly probability of changing status, and engineered 10-minute dropouts. The same generator fills the daily matrices used to train the CNN-LSTM and also produces the ground-truth availability during the simulated FL rounds. The forecaster is therefore tested almost exactly in-distribution: it only has to rediscover the rules baked into the synthetic data. Real-world availability has irregular, correlated, multi-scale structure; under distribution shift the eligibility matrix in Section VI-B3 will contain more false positives, scheduled clients will drop out, and the claimed 10-50% accuracy and test-loss gains over REFL could shrink or disappear. There is no oracle-availability or no-forecast ablation to show that forecasting, rather than the eligibility constraint, is the source of the improvement. If this concern lands, the central empirical contribution is not supported by the presented evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RIFLES, a framework for federated learning client selection that adds an availability-forecasting layer to a server-side scheduler. It formalizes client selection as the RIFLES scheduling problem, claims to prove NP-completeness via a reduction from resource-constrained scheduling, and introduces two scheduling heuristics (Greedy Heuristic and Least Recently Used) that operate on an eligibility matrix built from CNN-LSTM availability predictions and per-client response-time estimates. The authors evaluate the framework against Random, FedCS, and REFL on the WISDM and CIFAR-10 datasets using the FedScale simulator, reporting faster convergence, higher final accuracy, higher completion rates, and lower dropout rates. The paper claims improvements of 10–50% over baselines on a range of metrics, and positions itself as the first work to treat FL as a scheduling problem.","tokens_in":12918,"tokens_out":7359,"duration_ms":74357,"significance":"If the results hold, the idea of long-term availability forecasting to guide client scheduling would be a practically useful contribution to federated learning, particularly for cross-device settings with heterogeneous and intermittent client availability. The paper provides a clean conceptual pipeline (heartbeats, prediction matrix, eligibility matrix, scheduling policy) and its internal comparisons are consistent: both RIFLES variants outperform the three baselines on all reported metrics in the presented experiments. The work is also reproducible in structure: the framework is based on a public simulator (FedScale) and standard datasets, and the heuristic algorithms are described in moderate detail. However, the NP-completeness proof appears to contain a fundamental error, and the empirical evaluation relies on a synthetic availability generator that also trains the forecasting model, leaving the central contributions with limited support.","major_comments":[{"comment":"The reduction from RCS to RIFLES is invalid as presented. Setting α=100% and β=100% in the mapping strips the capacity constraint (ii), because 'no more than (α·n) clients' with α=1 is automatically satisfied, and it makes the per-job requirement (iii) vacuous, since 'at least (β·K) tasks' with β=1 resurrects only the trivial requirement that all K tasks be executed somewhere. The mapping also fails to account for the number of processors m in the RCS instance, which has no counterpart in the RIFLES definition. Moreover, the swap procedure to enforce condition (iv) is not feasibility-preserving: moving T_i^j from processor k to processor i displaces whatever task T_m^n was on processor i in that slot, and that task now violates condition (iv) on processor k. Thus the proof does not establish NP-hardness, and Theorem 1 is unsupported.","section":"Section V, Lemma 2"},{"comment":"The central empirical evaluation is carried out entirely on synthetic availability traces generated by the authors' own simulator, and the CNN-LSTM forecaster is trained on traces from that same generator. The model is therefore evaluated essentially in-distribution, and it is not surprising that it accurately predicts availability. There is no experiment with real client availability traces, no distribution-shift analysis (e.g., using a different generator or perturbed traces), and no ablation to separate the benefit of forecasting from the benefit of the eligibility constraint itself. For example, comparing RIFLES against a variant that uses ground-truth availability (oracle) or against a non-forecasting eligibility rule would indicate whether the 10–50% improvement comes from prediction or from the scheduling heuristic. Without such experiments, the claim that the forecasting layer is responsible for the reported gains is not established.","section":"Section VII-A2 and Section VIII"},{"comment":"The headline claim of 'improvement by between 10%-50% on a variety of metrics such as accuracy and test loss' is not consistent with the experimental tables. In the results text, the authors state that RIFLES maintains 'a 5–10% superior accuracy' over REFL and FedCS on the two datasets, which is a substantially smaller gain. The 10–50% figure appears to refer to other metrics (completion rate, dropout rate, lost time), but the paper does not provide a per-metric breakdown of percentage improvements, making the central quantitative claim ambiguous and difficult to verify.","section":"Abstract, Section VIII, Table II"},{"comment":"The Greedy Heuristic is not fully specified. The definition of 'unique clients' as those with |EligibleSlots_i| < α introduces a second, unrelated use of the symbol α, which was previously defined in Definition 1 as the global job selection proportion. In addition, the step 'we adjust the threshold or gap between rounds to maximize the participation of as many unique clients as possible' is described only qualitatively; it does not specify the adjustment procedure, its termination condition, or the interaction with the earlier gap constraint. This prevents the algorithm from being reproduced exactly from the text.","section":"Section VI-C1"}],"minor_comments":[{"comment":"The mapping table has typos: '17→r' should likely be '1→r' (one resource) and '1p→R_1(t)' is nonsensical as written; the intended meaning is not clear.","section":"Section V, Lemma 2 mapping"},{"comment":"The proof of Lemma 1 says 'checking conditions (i)-(v)' but Definition 1 lists only conditions (i)-(iv).","section":"Section V, Lemma 1"},{"comment":"The expected-update objective E[Δw] is defined but never used in the design, analysis, or evaluation of the proposed scheduler; its role in the paper is unclear.","section":"Section III"},{"comment":"The heartbeat loss threshold ε and the validity window W_i are introduced as assumptions but are never measured, varied, or stress-tested in the experiments, so their influence on the reported results is unknown.","section":"Section VI-A3"},{"comment":"The comparison set is limited to Random, FedCS, and REFL; more recent availability- or heterogeneity-aware schedulers such as Oort, TiFL, or FLASH are cited but not evaluated, which weakens the claim of superiority over the state of the art.","section":"Related Work and Experiments"},{"comment":"There are several grammatical issues (e.g., 'RIFLES provide significant improvement') and inconsistent notation (e.g., α used for two distinct concepts), which should be corrected in revision.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The NP-hardness proof is a clear mathematical error that must be addressed; the authors should either supply a valid reduction or substantially weaken the claim. The empirical evaluation also needs real-trace or robustness evidence to support the 10–50% improvement claim. I believe the core scheduling-with-forecasting idea is defensible and the experiments are internally consistent, so the paper could be salvageable with a major revision. However, if the authors cannot correct the complexity result or add meaningful external validation, I would consider rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's what you should know. RIFLES combines two things in a way I haven't seen before — heartbeat-derived availability forecasting (CNN-LSTM) feeding an eligibility-driven client scheduler for FL — and the system is written up clearly. But the NP-completeness proof in the theory section doesn't work, and the 10-50% gains are measured in a setting where the forecaster is trained on the same synthetic traces that generate the test availability. Real idea, unproven theory claim, unsecured empirical claim. It deserves a serious referee, but not acceptance as-is.\n\nThe good part is the system design. The daily availability matrix from heartbeats, the eligibility matrix that checks whether the predicted availability window covers expected response time plus a buffer, and the two scheduling policies (greedy with a round-gap and unique-client threshold; LRU) form a coherent, plausible pipeline. Related-work coverage is fair — REFL, FLASH, Oort, and TiFL are all there. The evaluation uses FedScale with device-speed and bandwidth heterogeneity from real measurements, two datasets, and sensible metrics (round-of-arrival, completion/dropout, unique participation). Within their synthetic world, the internal comparison is consistent: both variants beat Random, FedCS, and REFL.\n\nNow the soft spots, in proportion. The theory claim is broken, and I think it's worse than the reader's memo says. With α=β=100%, the resource constraint is gone and the RIFLES instance reduces to 'can each client's K unit tasks be placed into p slots with a per-slot cap' — a flow problem, not an NP-complete one. The swap argument also assumes a task has a 'correct' processor, which RCS never gives you. As defined, the problem looks polynomial, so the headline theory contribution is unsupported. That needs a real reduction or the claim has to go.\n\nThe empirical claim is the other load-bearing issue. The availability traces are entirely synthetic, and the CNN-LSTM is trained on the same generator it is later evaluated against — it mostly has to rediscover a 1.5x nighttime boost and a 20% hourly flip probability. Real availability is noisier and multi-scale; under distribution shift the eligibility matrix fills with false positives. There is no oracle-availability ablation and no no-forecast ablation, so you cannot tell whether forecasting or the eligibility/scheduling logic produces the gains. That is the single most important missing experiment. There is also no released code, only three baselines, and the heartbeat-loss threshold ϵ is never varied. The abstract's 10-50% is loose — the accuracy gaps in the figures are closer to 5-12% relative, with the bigger numbers coming from convergence speed and dropout.\n\nWho gets value: people working on cross-device FL client selection — the pipeline is a useful template even if these results don't carry. Send it to a serious referee; the direction is worth the reviewer hours. But my verdict on the current version is reject: fix the theory or cut it, add a real-trace or oracle experiment, and release the scheduler code.","headline":"A clearly-described FL scheduling pipeline worth a referee's time, but the NP-completeness proof doesn't hold and the empirical gains rest on synthetic traces; reject the current version and ask for real data plus a corrected theory.","tokens_in":13442,"tokens_out":12063,"would_cite":false,"duration_ms":115063,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RIFLES reframes federated-learning client selection as a scheduling problem, forecasts client availability from heartbeat signals, and reports 10-50% improvements in accuracy, test loss, and completion rates.","keywords":["federated learning","client selection","scheduling","availability forecasting","CNN-LSTM","heartbeat signals","NP-completeness","resource efficiency"],"falsifier":"Feed real device-availability traces collected from a deployed federated-learning app into the same RIFLES pipeline, run the same two benchmark tasks, and compare dropout and accuracy against random selection and the existing baselines; if forecast accuracy on the real traces is too low to keep dropout below the baseline's, the 10-50% improvement claim fails.","tokens_in":12455,"feed_emoji":"📶","tokens_out":8626,"duration_ms":80736,"temperature":0.7,"pith_summary":"Federated learning wastes rounds when the server picks clients that go offline mid-training. RIFLES tries to fix this by treating client selection as a scheduling problem: the server learns each client's likely availability from heartbeat signals, builds an eligibility matrix that says when a client can stay online long enough to finish a training task, and schedules training rounds at times and with clients that will actually complete. The paper formalises this scheduling problem and proves it NP-complete, so an efficient exact solver is not expected, and then offers two heuristic policies that run in a middleware layer. On two benchmark tasks, the authors report 10-50% improvements in accuracy, test loss, dropout, and completion rates over existing selection strategies, with one variant reaching 75% accuracy in 7 rounds on the activity-recognition benchmark.","feed_headline":"Scheduling by availability forecast lifts FL 10-50%","feed_subtitle":"RIFLES predicts client uptime from heartbeats and schedules training rounds when clients can actually finish.","key_machinery":"The central object is the eligibility matrix $E_i(s)$, built from the forecast availability matrix: $E_i(s)=1$ when the predicted remaining-availability window $\\Lambda_i^s$ from slot $s$ is at least the client's expected response duration $C_{\\mathrm{expected}}(i)$ plus a buffer $k$. This one inequality converts raw availability forecasts into actionable scheduling decisions and is what both scheduling policies optimise over. Around it sit the heartbeat-based daily availability matrices, the CNN-LSTM forecasting layer that predicts next-day availability from the recent daily matrices, and the formal reduction of the selection problem to resource-constrained scheduling, which supplies the NP-completeness result. The scheduling policies then select training slots with the most eligible clients while respecting a minimum gap between rounds; the least-recently-used variant breaks ties by choosing the most idle eligible clients.","core_discovery":"The central claim is that client availability is predictable enough to drive scheduling, and that scheduling on predicted availability makes federated learning substantially more efficient than round-by-round myopic selection. The paper defines a 'job' as a client's training workload, a 'task' as one local update, and a scheduling problem in which the server must choose, over a horizon, which clients run in which slots subject to participation proportions and a deadline; it proves this problem NP-complete by reduction from resource-constrained scheduling. To make scheduling tractable, RIFLES converts heartbeat-derived availability records into daily matrices, trains a CNN-LSTM to forecast next-day availability, and converts forecasts into an eligibility matrix by checking that each client's predicted continuous-availability window covers its expected response time plus a buffer. Two policies consume that matrix: a greedy heuristic that picks high-eligibility slots while enforcing a gap between rounds and favouring rarely-seen clients, and a least-recently-used variant that prioritises idle clients. The paper's evidence consists of emulated federated training on two datasets, where both variants outperform the baselines on accuracy, loss, dropout, completion, and participation diversity, with the reported 10-50% improvement range.","pith_inferences":["Editorial inference: the approach's value hinges on how well the CNN-LSTM forecasts generalise to real client populations; if real availability is aperiodic or heavily noisy, the eligibility matrix becomes unreliable and the schedule may degrade toward random selection.","Editorial inference: the heartbeat mechanism creates a design tension, because building the forecast requires clients to send heartbeats even when they are not training, and the paper assumes rather than measures that heartbeat loss stays below the stated threshold.","Editorial inference: because the eligibility check requires a predicted continuous window at least as long as expected response time, the method implicitly favours clients with long usage sessions; allowing preemptible or resumable local training could let shorter windows contribute.","Editorial inference: the 10-50% figure comes from emulated resource usage rather than measured energy or wall-clock time on physical devices, so a field test on real phones would be the decisive next experiment."],"forward_implications":["If availability is forecastable, the server can pre-plan a day's training rounds, avoiding the wasted computation and idle time caused by selecting clients that drop out.","The reported results imply that scheduling on predicted availability reaches target accuracy in fewer communication rounds, for example 75% accuracy in 7 rounds on the activity-recognition benchmark.","Lower dropout and higher completion rates mean less energy and bandwidth spent on updates that never get aggregated into the global model.","The NP-completeness result implies that no exact polynomial-time algorithm for the full scheduling problem is likely, so heuristic scheduling of this kind is a reasonable route.","The customisable middleware design means the same forecasting and eligibility pipeline can host scheduling policies beyond the two demonstrated variants."],"supporting_citations":[{"why":"It supplies the random-selection baseline that RIFLES must beat.","marker":"[5]"},{"why":"It supplies the capability-aware client-selection baseline that selects clients by estimated download, train, and upload times.","marker":"[14]"},{"why":"It supplies the availability-aware baseline that RIFLES compares against, along with the resource-efficiency framing used in the experiments.","marker":"[15]"},{"why":"It provides the heterogeneity-simulation approach for device speeds and communication bandwidths used in the experiments.","marker":"[22]"},{"why":"It supplies the simulation environment in which the compared methods are evaluated and resource usage is emulated.","marker":"[24]"},{"why":"It is the resource-constrained scheduling problem used as the target of the NP-completeness reduction.","marker":"[33]"},{"why":"It supplies the adaptive optimizer used for global model aggregation in the experiments.","marker":"[34]"}],"fun_headline_variants":["Forecast client uptime to schedule FL rounds, lift accuracy 10-50%","RIFLES predicts client availability to schedule training, boosting FL performance","Availability-aware scheduling makes federated learning more efficient","CNN-LSTM forecasts uptime to schedule FL clients, cutting resource waste","Scheduling FL on predicted availability yields 10-50% accuracy gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the synthetic availability patterns used to train the forecaster and run the experiments faithfully represent how real clients actually come and go; if real availability is less periodic or noisier than the generated patterns, the forecast quality that powers the eligibility matrix and both scheduling policies would degrade and the reported gains could vanish.","fun_headline_variants_meta":{"raw":{"variants":["Forecast client uptime to schedule FL rounds, lift accuracy 10-50%","RIFLES predicts client availability to schedule training, boosting FL performance","Availability-aware scheduling makes federated learning more efficient","CNN-LSTM forecasts uptime to schedule FL clients, cutting resource waste","Scheduling FL on predicted availability yields 10-50% accuracy gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000683,"raw_usage":{"total_tokens":3158,"prompt_tokens":1062,"completion_tokens":2096,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":2003}},"tokens_in":678,"tokens_out":2096,"duration_ms":13474,"temperature":1.0,"reasoning_tokens":2003,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:18:29.311247+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed real device-availability traces collected from a deployed federated-learning app into the same RIFLES pipeline, run the same two benchmark tasks, and compare dropout and accuracy against random selection and the existing baselines; if forecast accuracy on the real traces is too low to keep dropout below the baseline's, the 10-50% improvement claim fails.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"It supplies the random-selection baseline that RIFLES must beat."},{"cited_title":"Client selection for federated learning with heterogeneous resources in mobile edge,","cited_arxiv_id":null,"evidence_quote":"It supplies the capability-aware client-selection baseline that selects clients by estimated download, train, and upload times."},{"cited_title":"Refl: Resource-efficient federated learning,","cited_arxiv_id":null,"evidence_quote":"It supplies the availability-aware baseline that RIFLES compares against, along with the resource-efficiency framing used in the experiments."},{"cited_title":"Flash: Heterogeneity-aware federated learning at scale,","cited_arxiv_id":null,"evidence_quote":"It provides the heterogeneity-simulation approach for device speeds and communication bandwidths used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the resource-constrained scheduling problem used as the target of the NP-completeness reduction."}],"review_version":1}