{"id":"34aa2783-537f-462a-a565-5301e07f9906","arxiv_id":"2501.15949","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FedAvgOpt, a new aggregation strategy that optimizes per-client reweighting factors each round, reports improved accuracy and stability over FedAvg and five other baselines on brain MRI classification, but without error bars or repeated runs.","lead":"This paper proposes FedAvgOpt, a federated learning aggregation method that per round optimizes a reweighting factor for each client before averaging. In a brain MRI classification benchmark with limited data, it reports higher average accuracy than five standard aggregation strategies across four models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed FedAvgOpt superiority is statistically unsupported: one random split, no repeated seeds, and tiny margins for VGG16 and ResNet50V2.","rationale":"Good-faith reading: the paper proposes FedAvgOpt, a server-side reweighting heuristic that optimizes, via Nelder-Mead, a per-round objective measuring the closeness of a weighted aggregate to each client's local model. The central claim is explicitly empirical and limited to the studied use case (Section VII). For that claim to hold, the measured accuracies must rank strategies reliably, and the weakest link is the evidence base: exactly one random 20/80 split per client, one run per strategy/model, and no uncertainty quantification. Two of the four winning margins are below 0.5 percentage points, well within typical seed-to-seed variation for deep models on small data. The ad hoc, formally ambiguous objective in Eq. (2) is a real weakness, but it does not by itself falsify the empirical claim; conversely, a reproducible multi-seed advantage would make the heuristic worth reporting despite weak theory. Hence the load-bearing concern is statistical robustness rather than the lack of theoretical grounding. This aligns with the reader's conditional recommendation; I would keep the verdict CONDITIONAL, with the required revisions being multi-seed evaluation, confidence intervals, and a clarification of the objective's notation. I see no grounds for rejection, since the method is described sufficiently for replication in principle and the results, if confirmed, would be useful for practitioners in the federated medical-imaging setting.","tokens_in":10522,"tokens_out":4840,"duration_ms":47797,"concrete_test":"Rerun the complete evaluation (all 6 aggregation strategies × 4 base models, keeping the same hyperparameters and data distribution) with at least 10 independent random seeds / 10 different 20/80 stratified splits. For each strategy and model, report the mean and 95% confidence interval (or paired bootstrap) of the 10-round average test accuracy. Check whether FedAvgOpt remains strictly best in all four models and whether the VGG16 and ResNet50V2 margins from Table II (0.27 pp and 0.49 pp) stay positive across all seeds; if either margin flips sign in more than 5% of seeds or bootstrap resamples, the central claim of consistent superiority is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim ('FedAvgOpt is the most stable aggregation strategy of the 6 analyzed across the 4 base models', Section VII) rests on Table II, which reports mean test accuracy over ten FL rounds from a single random 20/80 train/test split per client, with no repeated seeds, confidence intervals, or significance tests. For two of the four models the winning margins are very small: VGG16 0.77010 vs 0.76742 for FedAvgM (+0.27 pp) and ResNet50V2 0.79142 vs 0.78652 for FedMedian (+0.49 pp). Training is stochastic and the data split is random, so these differences are within plausible run-to-run variation. The ad hoc nature of the FedAvgOpt objective (Eq. 2) and the ambiguous 'w_FedAvg x' notation are genuine weaknesses, but they would be mitigated if the empirical ranking were reproducible across seeds; the converse is not true. Thus the least secure load-bearing condition is statistical: the observed superiority of FedAvgOpt may be a random artifact of one particular split and initialization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedAvgOpt, a server-side federated learning aggregation strategy that reweights each client's contributed model by a per-client scalar alpha. The alphas are obtained by minimizing a heuristic objective f(x) defined in Eq. (1) as the sum, over clients, of ratios of squared distances between a reweighted average model and each local model, using the Nelder-Mead method from the initial point x0 = 1. The method is evaluated on a brain MRI classification task with four clients, four pretrained CNN base models (VGG16, InceptionV3, ResNet50V2, DenseNet121), and five baseline aggregation strategies (FedAvg, FedAvgM, FedMedian, FedOpt, FedYogi), under a simulated limited-data setting with 20% of each client's data used for training. The paper reports mean aggregated test accuracy over ten federated rounds and concludes in Section VII that FedAvgOpt is the most stable aggregation strategy of the six analyzed across the four base models.","tokens_in":10743,"tokens_out":4716,"duration_ms":49832,"significance":"If the empirical claim were firmly established, FedAvgOpt would be a modest but potentially useful contribution: it is a server-side-only aggregation rule that requires no client-side modifications, which is practically attractive for privacy-sensitive medical deployments. The experimental design is transparent in several respects: it uses an open dataset, four architectures with very different parameter counts and depths, six aggregation strategies, and a clearly stated pseudocode for the proposed method. However, the central conclusion currently rests on a single experimental realization with no repeated seeds, no confidence intervals, and no significance testing, and the reported performance metric (mean accuracy over rounds) can favor a strategy that starts well in round 1 even if its final-round performance is not superior. The optimization objective is also introduced heuristically with no theoretical link to generalization or convergence. These issues are load-bearing because the paper's main contribution is its empirical ranking of aggregation strategies; the ranking may change under repeated runs or under a different evaluation metric.","major_comments":[{"comment":"The central claim that FedAvgOpt is 'the most stable aggregation strategy' is not statistically supported. The experiments appear to use a single random stratified client partition and a single random 20/80 train/test split per client, with no repeated seeds, no confidence intervals, and no significance tests. For two of the four base models the reported margins over the best baseline are very small: VGG16 0.77010 vs. 0.76742 for FedAvgM (+0.27 percentage points) and ResNet50V2 0.79142 vs. 0.78652 for FedMedian (+0.49 percentage points). Training is stochastic and the data split is random, so these differences are within plausible run-to-run variation. The authors should repeat the entire protocol with multiple seeds and report mean and standard deviation (or confidence intervals) for each strategy and model, and ideally perform a paired significance test across seeds.","section":"Section VI, Table II, Section VII"},{"comment":"The primary reported metric, 'mean aggregated accuracy obtained in the 10 rounds,' is not a standard measure of the quality of the final federated model. Averaging over rounds mixes early-round behavior with convergence behavior, and the paper itself notes that FedAvgOpt starts from a higher first-round accuracy for VGG16, DenseNet121, and ResNet50V2. A strategy that has a high initial accuracy but converges to the same or lower final accuracy can therefore look better under this metric. The authors should additionally report the final-round test accuracy and per-round learning curves with variability, or clearly justify why mean-over-rounds is the appropriate target metric for the claimed convergence improvement.","section":"Section VI, Table II, Figure 4"},{"comment":"The FedAvgOpt objective is ad hoc: there is no theoretical argument or prior evidence that minimizing the sum of ratios of squared distances between a reweighted average and each local model improves the global model's generalization or convergence. In addition, the alpha_i are optimized without any normalization or non-negativity constraint, so the resulting w_FedAvgOpt is not in general a convex combination of the client models; the paper does not discuss whether negative or large alpha values can occur or what effect they have on the aggregated model. The notation in Eq. (2), 'w_FedAvg x,' is also ambiguous: w_FedAvg appears to be a vector of aggregated parameters, while x is the alpha vector, so the expression should be written as the weighted average (1/sum n_i) * sum_i w_i n_i x_i rather than as a product of w_FedAvg and x. The authors should clarify the formulation, add constraints or justify their absence, and discuss the behavior of the Nelder-Mead optimization from the single starting point x0 = 1.","section":"Section IV, Eqs. (1)-(2), Algorithm 1"},{"comment":"The local training configuration is incompletely specified: the paper gives the optimizer, learning rate, batch size, and number of rounds, but does not state the number of local training epochs (or local steps) per client per round. Local epochs are a critical hyperparameter in federated learning and directly affect the comparison between aggregation strategies. Without this information, the experiments are not fully reproducible, and the observed ranking could depend on an undocumented choice. The authors should report the number of local epochs/steps for each base model and, if varied, justify the choice.","section":"Section V, Section V.C"}],"minor_comments":[{"comment":"There is a typo in the Introduction: 'psedudonymization' should be 'pseudonymization.'","section":"Section I"},{"comment":"There are typos in the results discussion: 'mote than 10%' should be 'more than 10%,' and 'FedAvgOot' should be 'FedAvgOpt.'","section":"Section VI"},{"comment":"In the ResNet-50 V2 description, '128 sy neurons' appears to be a typo for '128 neurons.'","section":"Section V.B"},{"comment":"The notation 'w_FedAvg x' in Eq. (2) should be explicitly defined as the weighted average of client weights with coefficients x_i; as written, it looks like a matrix-vector product and is inconsistent with Eq. (1).","section":"Section IV, Eq. (2)"},{"comment":"The paper should state the random seed or seeds used for the client partition and the train/test split, since the entire evaluation is based on a single split.","section":"Section V.A"},{"comment":"The descriptions of FedAvgM, FedMedian, and FedOpt would benefit from explicit references to the exact algorithm numbers and equations in the cited papers, since the current text summarizes rather than reproduces the update rules.","section":"Section II"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical application note with a useful practical angle (server-side-only aggregation for privacy-sensitive medical imaging). The main risk is not circularity or fabricated results but insufficient statistical support for the headline ranking. I would be willing to reconsider after the authors add multi-seed experiments, report final-round metrics alongside mean-over-rounds, and clarify the objective and aggregation constraints. The missing local-epoch specification should also be fixed, as it currently prevents independent reproduction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is FedAvgOpt: per-client reweighting factors, alpha, tuned each round by Nelder-Mead against an objective that pulls the aggregated model toward every client model. That is a genuine, if modest, addition to the FedAvg family. The paper also does something right: it compares against five established baselines on four architectures, uses a real medical imaging task, and does not oversell the theory. There is no circularity; alpha is fit on client models, not test data, and the method is not dressed up as something it is not.\n\nWhere it falls short is statistics. The main claim — FedAvgOpt is the most stable and best aggregator — rests on one random 20/80 split per client, ten rounds, and no repeated seeds or confidence intervals. For VGG16 the margin over FedAvgM is 0.27 percentage points; for ResNet50V2 it is 0.49 points over FedMedian. With stochastic training and a single split, those gaps are indistinguishable from run-to-run noise. The larger margins on InceptionV3 and DenseNet121 are more suggestive, but without seeds they do not justify the strength of the conclusion. The ad hoc objective (Eq. 2) and the non-normalized weighting in the aggregation formula are also genuine weaknesses, and the notation around w_FedAvg x is sloppy enough to slow a careful reader down.\n\nStill, the central idea is not wrong. It is an incremental heuristic, presented as such in most of the text, and the experiments, while under-powered, are not fabricated. The paper would be substantially improved by adding multiple seeds, reporting standard deviations, testing a non-i.i.d. split, and cleaning up the math. I would not accept it as is, but I would not desk reject it either. It deserves a serious referee and a request for major revision. The right audience is someone working on practical FL aggregation for medical imaging, not someone looking for new theory. I would not cite it in my own work until the empirical claims are backed by proper uncertainty quantification.","headline":"A plausible but statistically under-supported heuristic aggregation rule; the empirical ranking could easily be noise, though the method is honest and worth a careful revision, not a desk reject.","tokens_in":11254,"tokens_out":789,"would_cite":false,"duration_ms":9488,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces FedAvgOpt, a server-side aggregation strategy that reweights each client's model each round by solving a small nonlinear optimization, and reports that it beats five classical aggregation strategies in mean accuracy…","keywords":["federated learning","aggregation strategy","FedAvgOpt","brain MRI classification","limited data","convergence","medical imaging","server-side optimization"],"falsifier":"Run the same four-client, four-backbone protocol with five or more different random 20/80 split seeds and report whether FedAvgOpt still achieves the highest mean accuracy in every case; if any other strategy surpasses it on average, the claim that it is the most stable across all four backbones would fail. Alternatively, construct a non-i.i.d. split where FedAvgOpt underperforms FedAvg by a clear margin.","tokens_in":10321,"feed_emoji":"🧠","tokens_out":4773,"duration_ms":42923,"temperature":0.7,"pith_summary":"The paper proposes a new federated learning aggregation strategy, FedAvgOpt, and tests it against five classical strategies on a four-client brain MRI classification task with limited training data. The central claim is that FedAvgOpt achieves the highest average test accuracy over ten rounds for all four pretrained CNN backbones considered (VGG16, InceptionV3, ResNet50V2, DenseNet121). The authors interpret this as better and more stable convergence, and note that FedAvgOpt requires no client-side changes beyond sending local model weights. This matters because aggregation is the key privacy-preserving fusion step in federated learning, and improved aggregation could support collaboration between medical centers without sharing imagery.","feed_headline":"New aggregation strategy tops five federated-learning baselines","feed_subtitle":"In 10-round brain-MRI tests with four backbones and limited data, FedAvgOpt posts the best mean accuracy every time.","key_machinery":"The load-bearing object is the per-round weight vector $\\alpha \\in \\mathbb{R}^n$ obtained by minimizing the objective $$f(x) = \\sum_{j=1}^n \\frac{\\|w_{\\mathrm{FedAvg}} x - w_j\\|^2}{\\|w_{\\mathrm{FedAvg}} x + w_j\\|^2},$$ where $w_{\\mathrm{FedAvg}}$ is the data-count-weighted average of the local model weights $w_j$ from the $n$ clients. The aggregated model then becomes $w_{\\mathrm{FedAvgOpt}} = \\frac{1}{\\sum_i n_i} \\sum_i n_i w_i \\alpha_i$, so that $\\alpha = (1,\\ldots,1)$ reduces exactly to FedAvg. The server solves this unconstrained nonlinear problem with a standard derivative-free simplex method initialized at the all-ones vector, and the reweighted average is passed to clients as the next round's global model.","core_discovery":"The central claim is that FedAvgOpt, a server-side aggregation function that reweights client models each round by solving a small optimization problem, outperforms FedAvg, FedAvgM, FedMedian, FedOpt, and FedYogi in mean aggregated test accuracy over ten federated rounds, across all four base models studied. Concretely, Table II reports FedAvgOpt reaching 0.82209 mean accuracy with DenseNet121 versus 0.79740 for FedMedian, and 0.70999 versus 0.67959 with InceptionV3. The paper also reports that FedAvgOpt's per-round accuracy curves are more stable, with fewer sharp drops, and that in three of the four backbones it achieves the highest first-round accuracy, exceeding the others by more than ten percentage points.","pith_inferences":["A natural extension is to test FedAvgOpt under non-i.i.d. client distributions, since the current experiments use an i.i.d. stratified split and the paper explicitly lists this as future work; the mechanism may behave differently when client data is heterogeneous.","The proposed objective is an ad hoc ratio of distance terms without a theoretical convergence or generalization guarantee, so a plausible falsifying scenario is a data split or base model where the extra optimization actually degrades accuracy relative to plain FedAvg.","Because the ranking rests on a single random 20/80 split per client with no repeated seeds, an editorial caveat is that the reported order of strategies could be partly an artifact of that particular split; repeated trials would be needed to confirm the ordering.","The ratio structure resembles a normalized distance that may relate to cosine-style similarity between weight vectors, which could offer a bridge to theoretical analysis of FedAvgOpt as a form of robust mean estimation."],"forward_implications":["Switching the server-side aggregator from FedAvg to FedAvgOpt improves mean accuracy over ten rounds for all four tested backbones in the studied i.i.d. setting, with gains of roughly three accuracy points for InceptionV3 and DenseNet121.","FedAvgOpt requires no client-side modifications and no additional communication beyond the local model weights, so it can be dropped into existing federated learning pipelines.","In three of the four backbones (VGG16, ResNet50V2, DenseNet121), FedAvgOpt achieves the highest first-round accuracy, indicating faster initial convergence.","For an unstable base model such as InceptionV3, FedAvgOpt's per-round curve avoids the significant accuracy drops seen with FedAvg, FedAvgM, FedOpt, and FedYogi.","The method is tested with only 20% of each client's data used for training, so the reported improvements appear specifically in the limited-data regime the paper targets."],"supporting_citations":[{"why":"Defines FedAvg, the baseline aggregation strategy that FedAvgOpt is designed to improve.","marker":"[5]"},{"why":"Provides the FedAvgM momentum-based aggregation strategy compared as a baseline.","marker":"[12]"},{"why":"Provides the FedMedian byzantine-robust aggregation strategy used as a baseline.","marker":"[13]"},{"why":"Provides FedOpt and FedYogi adaptive federated optimization strategies compared as baselines.","marker":"[14]"},{"why":"Supplies the brain MRI dataset used to construct the four-client federated classification task.","marker":"[26]"},{"why":"Implements the federated learning orchestration used to run the aggregation comparisons.","marker":"[11]"}],"fun_headline_variants":["FedAvgOpt beats five baselines in federated learning","New aggregation rule improves federated learning convergence","Limited data? FedAvgOpt still converges faster","Brain MRI test: new federated aggregation wins","Optimizing aggregation: better federated learning with less data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that minimizing the ad hoc ratio-sum objective f(x) yields aggregation weights that generalize better, without a theoretical analysis of why that objective relates to generalization; the experiments also rely on a single 20/80 stratified split per client, so the ranking could depend on that particular split.","fun_headline_variants_meta":{"raw":{"variants":["FedAvgOpt beats five baselines in federated learning","New aggregation rule improves federated learning convergence","Limited data? FedAvgOpt still converges faster","Brain MRI test: new federated aggregation wins","Optimizing aggregation: better federated learning with less data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1627,"prompt_tokens":915,"completion_tokens":712,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":638}},"tokens_in":531,"tokens_out":712,"duration_ms":6175,"temperature":1.0,"reasoning_tokens":638,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:49:27.486988+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four-client, four-backbone protocol with five or more different random 20/80 split seeds and report whether FedAvgOpt still achieves the highest mean accuracy in every case; if any other strategy surpasses it on average, the claim that it is the most stable across all four backbones would fail. Alternatively, construct a non-i.i.d. split where FedAvgOpt underperforms FedAvg by a clear margin.","supporting_citations":[],"review_version":1}