{"id":"0403f73a-b7ec-4344-9223-6754ce71fba2","arxiv_id":"2412.19987","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"DPGA combines delayed gradient averaging with randomly sampled partial-gradient masks, and its CIFAR-10/100 experiments report much higher accuracy and lower communication than FedAvg, DGA, and LG-Fed.","lead":"Federated learning clients normally send full model updates and then wait for the server, which wastes bandwidth and idle compute. This paper proposes DPGA, where each client sends only a randomly chosen part of its gradient and keeps training locally while that partial update travels to the server and back.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is unsupported as written: Algorithm 1 and Eq. (10) do not define a consistent delayed-correction rule, and no convergence or bias analysis is provided; until the algorithm is specified or reproduced, the reported Table I margins cannot be attributed to DPGA.","rationale":"The paper proposes DPGA as a communication-efficient FL method with a delayed partial-gradient correction. The strongest claim is purely empirical, so the burden rests on the correctness of the described algorithm and the reproducibility of the experiments. Reading in good faith, I see a genuine soft spot: the derivation in Eq. (10) is the only justification for the correction, and it is not supported by any convergence analysis; more importantly, Algorithm 1's line 6 is not a faithful transcription of that derivation. The branch condition, the use of z^{t+D}_i at k=1, and the mask arithmetic in Eq. (6) all introduce ambiguities that make it impossible to determine from the manuscript what update was actually executed in Table I. This is a mechanical and textual problem, not an accusation about the authors. The reader's weakest-assumption analysis identifies the same delayed-correction mechanism, and my concrete check would settle whether the algorithm as written is internally consistent. I therefore see no reason to change the reader's REJECT verdict; a revised version with a clean derivation, a consistent pseudocode, and full experimental details could change that outcome.","tokens_in":50,"tokens_out":10808,"duration_ms":169268,"concrete_test":"Take t=1, D=2, K=1 and write down the state at the start of round 3 using the notation of Section II-C. Evaluate the target state from Eq. (10), namely w^0 - eta(z^2_1 + z^1_{1,pers} + g^1(p)), and evaluate the state produced by executing Algorithm 1, line 6 on the same initialization. If the two states differ at machine precision, the paper does not define the algorithm that produced Table I, and the authors should be asked for the exact hyperparameters and code to rerun the IID CIFAR-10 row.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: DPGA 'consistently outperforms' FedAvg/DGA/LG-Fed in accuracy, communication cost, and run time (Section IV). The load-bearing mechanism is the delayed partial-gradient correction: Eq. (10) and Algorithm 1, line 6 assume that after D extra local SGD rounds, the model can be corrected by replacing the stale shared part z^t_i(p^t) with the newly arrived global shared gradient g^t(p^t). This replacement is unbiased only under a locally linear/quadratic model of the loss; for non-convex neural networks no convergence or bias analysis is provided, so there is no reason from the text that the correction preserves accuracy. More concretely, the pseudocode does not follow from Eq. (10): the special branch at line 6 subtracts the full-round accumulated gradient z^{t+D}_i at the first local epoch k=1, before that round's gradients have been computed, and the trigger t mod D = 1 fires only every D rounds, whereas a delayed gradient from round t should arrive at the start of round t+D for every t. Eq. (6) also writes the merge of global shared and local personal gradients as g^t + g^t_i(1-p^t), which is not well-defined; the correct component-wise expression is g^t \\odot m^t_i + g^t_i \\odot (1-m^t_i). Because no code, hyperparameters, or seed statistics are released, the reported margins (e.g., 93.08% vs 60.08% in Table I) cannot be attributed to a well-defined algorithm; if the correction rule is mis-specified, the parallel-computation benefit and the accuracy gains collapse.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Delayed Random Partial Gradient Averaging (DPGA), a federated learning algorithm that combines partial gradient exchange (with an update rate sampled by a random walk) and delayed gradient averaging so that clients can compute locally while gradients are in transit. The authors claim that DPGA simultaneously reduces communication bandwidth and latency, and report experiments on CIFAR-10/100 showing that DPGA \"consistently outperforms\" FedAvg, DGA, and LG-Fed in accuracy, communication cost, and run time.","tokens_in":7789,"tokens_out":4091,"duration_ms":41038,"significance":"If substantiated, the core idea of overlapping local computation with partial-gradient communication would be a useful contribution to communication-efficient federated learning, and the reported accuracy margins are striking. However, the manuscript does not establish the claim: the algorithm is not specified consistently, the delayed-correction mechanism is not analyzed, the experiments are not reproducible from the information given, and the baseline set is too narrow to support the \"state-of-the-art\" characterization. The paper also ships no code, no hyperparameters, no error bars, and no convergence or bias analysis, so the significance is currently only potential.","major_comments":[{"comment":"The delayed-correction rule is not well defined, and the pseudocode does not follow from Eq. (10). In Eq. (8), z^t_i is defined as the full accumulated local gradient sum_{k=1}^K g^{t,k}_i, but later it is designated as the shared-part accumulated gradient sum g^{t,k}_i(p^t_i), so the notation is overloaded. More seriously, Algorithm 1's special branch at line 6 subtracts z^{t+D}_i at the first local epoch k=1, before the gradients of that round have been computed; the trigger 't mod D = 1' fires only every D rounds, whereas a delayed gradient from round t should arrive at the start of round t+D for every t. As written, the algorithm cannot be implemented unambiguously.","section":"Section II-C, Eq. (10) and Algorithm 1"},{"comment":"Equation (6) writes the merge of global shared and local personal gradients as g^t + g^t_i(1-p^t_i), which is not a well-defined vector expression because g^t is a masked gradient and (1-p^t_i) is a scalar/ratio, not a mask. The intended component-wise operation is g^t \\odot m^t_i + g^t_i \\odot (1-m^t_i), with m^t_i the binary mask. This ambiguity affects the core definition of the exchanged gradients and the reproducibility of the method.","section":"Section II-B, Eq. (6)"},{"comment":"The experimental evidence is not reproducible. The manuscript reports no values for the learning rate η, local epochs K, latency D, batch size, number of communication rounds T, or random seeds, and it gives no error bars or variance across runs. Tables I and II report single accuracy numbers, and Figure 2 plots 'Best test accuracy' without defining the selection rule. Without these details, the reported margins (e.g., Table I: DPGA 93.08% vs. FedAvg 60.08% on IID CIFAR-10) cannot be attributed to a specific algorithm implementation.","section":"Section III, Tables I-II and Figure 2"},{"comment":"The comparison set is too narrow to support the claim of outperforming state-of-the-art methods. The introduction cites FedMask, CD2-pFed, masked training with partial gradients, and gradient compression as related work on partial-parameter exchange, but none of these are included as baselines. The paper compares only against FedAvg, DGA, and LG-Fed, so the central empirical claim is not demonstrated against the methods the paper itself identifies as the relevant state of the art.","section":"Section III, baselines"},{"comment":"The delayed-correction step is justified only by a heuristic derivation that replaces a stale shared gradient with the newly arrived global shared gradient. For non-convex loss landscapes of deep networks, there is no convergence or bias analysis showing that this substitution preserves the optimization trajectory. Since the entire accuracy and run-time gain rests on this correction, the absence of any theoretical or empirical validation of the correction's bias is a load-bearing gap.","section":"Section II-C, Eq. (9)-(10)"}],"minor_comments":[{"comment":"The text contains several typographical and formatting issues, including the title spacing 'A VERAGING' and inconsistent use of superscripts and subscripts for local model weights (e.g., w^t_i vs. w^n).","section":"General"},{"comment":"The binomial coefficient notation in Eq. (4) is garbled ('C y x = y!/(x!(y-x)!)'), and the transition probabilities are not fully specified: the number of random-walk steps m, the step size 0.1, and the constant A=10 are introduced without explanation or sensitivity analysis.","section":"Section II-B, Eq. (4)"},{"comment":"The initial update rate p^0 and the exact sampling procedure for p^t from the random walk are not defined, and the claim that the transition probabilities are normalized (Eq. (5)) is not accompanied by the actual normalization constant.","section":"Section II-B, random walk"},{"comment":"The pseudocode has structural problems: line 5 states a condition ('When the delayed global gradient is received') but the actual conditional appears only in line 6, and the lines inside the client loop do not nest correctly with the 'Send' instruction at line 8.","section":"Algorithm 1"},{"comment":"The y-axis labels of Figure 2 read 'Best test accuracy (%)'; the paper should specify whether these are the best accuracy over the training trajectory or the final accuracy, and how the 'communication time' and 'communication parameters' were measured for each method.","section":"Section III, Figure 2"}],"recommendation":"reject","confidential_remarks":"The manuscript is a single-author preprint with a potentially interesting idea, but the technical presentation is not at the level required for a peer-reviewed venue. The algorithm specification is internally inconsistent, the delayed-correction mechanism is unanalyzed, and the experiments are not reproducible from the provided information. The lack of code and hyperparameters, combined with the absence of relevant baselines cited in the paper's own introduction, makes the central empirical claim unsupported. I would not recommend major revision because the core algorithm as written is not well defined; any meaningful revision would require a substantially rewritten method description and a new experimental study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper marries delayed gradient averaging with partial gradient exchange and a random-walk-controlled update rate. The motivation is real, the combination is new, and the reported numbers would be significant if true. But as written, the algorithm is not fully specified, the correction-step derivation is internally inconsistent, and there is no code or experimental detail to back the claims. I would not trust any of the accuracy margins until the method is pinned down.\n\nWhat's actually new: DGA already delays averaging, and FedMask, LG-Fed, and masked training already send partial gradients. DPGA's only new ingredient is the random-walk update-rate scheduler plus the specific combination. That is a legitimate incremental idea, not a new principle. The paper does engage the right baselines (FedAvg, DGA, LG-Fed) and covers several heterogeneity settings on CIFAR-10/100, which is a real effort.\n\nThe soft spots are serious. Eq. (6) writes the merged gradient as g^t + g^t_i(1-p^t), which is not well-defined for vectors; the correct expression is a component-wise mask. Eq. (10) derives the delayed correction by telescoping local updates from a fixed w0, but in the actual algorithm clients are updating on their own trajectories, and no convergence or bias argument is given for non-convex losses. Algorithm 1's special branch triggers only when t mod D = 1 and k=1, but a delayed gradient from round t should arrive at the start of round t+D for every t; the line that subtracts z^{t+D}_i also fires before that round's gradients exist. So the pseudocode does not implement the text's correction rule.\n\nThe experiments are the entire evidence base, and they ship no code, no hyperparameters (K, eta, D, batch size, rounds, seeds), no repeats, no error bars. The 30+ point accuracy gains over FedAvg, DGA, and LG-Fed on CIFAR-10 are outside the normal range and unexplained. The figure caption saying LG-Fed is \"fine-tuned based on the FedAvg pre-trained model\" adds another ambiguity.\n\nWho this is for: someone working on communication-efficient FL might want to know about the random-walk scheduler idea, but I would not build on this paper's numbers.\n\nRecommendation: I would send it to a referee rather than desk-reject, because the problem is important and the combination is worth examining, but the referee should require a complete algorithm specification, a clean derivation, and full experimental details. In current form it should not be accepted.","headline":"Plausible combination of delayed averaging and partial gradients, but the algorithm is not fully specified and the experiments are not reproducible; the reported gains cannot be trusted as written.","tokens_in":8355,"tokens_out":4859,"would_cite":false,"duration_ms":43676,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that delayed random partial gradient averaging solves both bandwidth and latency bottlenecks in federated learning, reporting large accuracy and communication gains over FedAvg, DGA, and LG-Fed.","keywords":["federated learning","communication efficiency","partial gradient averaging","delayed aggregation","random walk update rate","non-IID data","CIFAR-10","CIFAR-100"],"falsifier":"Run the reported CIFAR-10 non-IID comparison with the delayed-correction term in Eq. (10) disabled and all other settings identical: if accuracy stays near DPGA's reported 80-90%, the correction is not the source of the gains; if it drops toward the FedAvg and DGA baseline near 60%, the unproved correction is the load-bearing ingredient and would need a formal convergence or bias analysis.","tokens_in":7192,"feed_emoji":"📡","tokens_out":7363,"duration_ms":70517,"temperature":0.7,"pith_summary":"The paper proposes a federated learning method, Delayed Random Partial Gradient Averaging (DPGA), that tries to remove both communication bottlenecks at once: bandwidth and latency. In DPGA, each client uploads only a randomly chosen fraction of its local gradient, with the fraction set by a random-walk update rate, and keeps doing local computation while that partial gradient travels to the server. When the averaged global partial gradient arrives, the client substitutes it for the stale shared part of its accumulated local gradient. The reported experiments on CIFAR-10 and CIFAR-100, under IID and non-IID data, claim DPGA beats FedAvg, DGA, and LG-Fed in test accuracy, communication cost, and run time; for example, IID CIFAR-10 accuracy is reported as 93.08% for DPGA versus 60-62% for the three baselines. If true, this would make DPGA a drop-in, communication-efficient replacement for standard federated averaging.","feed_headline":"Delayed partial gradients lift federated accuracy by 33 points","feed_subtitle":"DPGA shares only part of each client's gradient and computes while communicating, beating FedAvg, DGA, and LG-Fed on CIFAR-10/100.","key_machinery":"The mechanism is the DPGA update, which combines two ideas. First, a one-dimensional random walk over the update rate $p_t$ in $(0,1]$ decides in each communication round what fraction of each local gradient is shared and therefore uploaded; Top-K sparsification fixes which components are shared, and the server aggregates component-wise over clients whose masks contain each component. Second, delayed partial gradient averaging lets clients keep training for $D$ local rounds while the partial global gradient is in flight; when the global partial gradient $g^t(p^t_i)$ arrives, the client replaces the stale shared part $z^t_i(p^t_i)$ of its accumulated gradient with the new global one, as in Eq. (10). This correction is what allows computation and communication to overlap without simply discarding the delayed information.","core_discovery":"The paper's claim, stated on its own terms, is that sharing only part of each local gradient while overlapping communication with local computation consistently outperforms the state-of-the-art methods in accuracy, communication cost, and run time. The evidence is empirical: on IID CIFAR-10, DPGA reaches 93.08% test accuracy against 60.08% for FedAvg, 61.02% for DGA, and 62.14% for LG-Fed; on non-IID CIFAR-10 with the strongest skew it reports 86.01% against under 60% for all baselines; on CIFAR-100 it reports 71.22% IID accuracy and 80.06% under heavy non-IID skew, where the best baseline reaches 54.29% and 50.39%. The authors further report that reaching a target accuracy requires only a small fraction of the communication parameters and communication time of the alternatives. The paper gives no convergence theorem; the discovery is the experimental result that a delayed, randomly masked partial-gradient averaging rule improves accuracy while reducing communication.","pith_inferences":["The paper leaves the delayed-correction rule in Eq. (10) without a bias or convergence analysis; a formal treatment of this term is the most direct way to tell whether the accuracy gains are a genuine optimization effect or an artifact of the specific experiments.","The random-walk scheduler is not obviously load-bearing: a fixed or monotonically decaying update rate might produce similar results, and ablating the scheduler would separate the core partial-plus-delayed idea from the way $p_t$ is chosen.","The reported margins over FedAvg are large enough that a reimplementation with full hyperparameter values, multiple random seeds, and identical compute budgets would be needed before relying on the numbers; the paper does not report these details.","An immediate extension would allow the transmission delay $D$ to differ per client, reflecting heterogeneous bandwidth; the paper assumes all clients share the same delay, which real systems rarely satisfy."],"forward_implications":["If the reported results are correct, an FL system can reach a given target accuracy while transmitting only a fraction of each gradient, cutting uplink and downlink bytes roughly in proportion to the update rate $p_t$.","Because clients compute during communication, wall-clock training time should shrink by nearly the communication latency, without sacrificing the accuracy gains.","The dynamic update rate gives a tunable knob between personalization and shared-model convergence: smaller $p_t$ shares less of the model and larger $p_t$ shares more.","The method applies to standard deep networks and standard local SGD, so it can be added to an existing FedAvg-style pipeline without changing the objective or the local optimizer."],"supporting_citations":[{"why":"Defines federated averaging, the baseline global update rule and the main comparison method.","marker":"[1]"},{"why":"Introduces delayed gradient averaging, the latency-reduction baseline that DPGA extends by adding partial gradients.","marker":"[5]"},{"why":"Introduces LG-Fed, the partial-parameter-upload baseline that DPGA is compared against.","marker":"[9]"},{"why":"Supplies the random-walk model used to sample the dynamic update rates $p_t$.","marker":"[14]"},{"why":"Supplies Top-K sparsification, which determines which gradient components are shared in each round.","marker":"[17]"},{"why":"Defines the Dirichlet-based non-IID data partition used in the CIFAR-10/100 experiments.","marker":"[21]"}],"fun_headline_variants":["Partial gradient sharing lifts federated accuracy to 93%","Delayed partial gradients beat full sync in federated learning","Federated learning: share less, compute while communicating, win","Sparse gradient averaging improves federated accuracy and speed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result depends on the delayed-correction step in Eq. (10): after $D$ extra local rounds, replacing the stale shared part of the accumulated gradient with the newly arrived global partial gradient must improve, or at least not corrupt, the update; the paper provides no convergence or bias argument for this assumption.","fun_headline_variants_meta":{"raw":{"variants":["Partial gradient sharing lifts federated accuracy to 93%","Delayed partial gradients beat full sync in federated learning","Federated learning: share less, compute while communicating, win","Sparse gradient averaging improves federated accuracy and speed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1607,"prompt_tokens":954,"completion_tokens":653,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":585}},"tokens_in":570,"tokens_out":653,"duration_ms":6409,"temperature":1.0,"reasoning_tokens":585,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:41:48.068385+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the reported CIFAR-10 non-IID comparison with the delayed-correction term in Eq. (10) disabled and all other settings identical: if accuracy stays near DPGA's reported 80-90%, the correction is not the source of the gains; if it drops toward the FedAvg and DGA baseline near 60%, the unproved correction is the load-bearing ingredient and would need a formal convergence or bias analysis.","supporting_citations":[{"cited_title":"Fedcorr: Multi- stage federated learning for label noise correction,","cited_arxiv_id":null,"evidence_quote":"Defines the Dirichlet-based non-IID data partition used in the CIFAR-10/100 experiments."},{"cited_title":"Delayed gradient averaging: Tolerate the communication latency for federated learning,","cited_arxiv_id":null,"evidence_quote":"Introduces delayed gradient averaging, the latency-reduction baseline that DPGA extends by adding partial gradients."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the random-walk model used to sample the dynamic update rates $p_t$."}],"review_version":1}