{"id":"13f1f73b-2382-40df-9f8e-be5d96e57f84","arxiv_id":"2412.11408","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FedSB combines client-level label smoothing with equal-sized per-client training budgets and reports state-of-the-art accuracy on three of four federated domain generalization benchmarks.","lead":"This paper proposes FedSB, a federated learning method that softens training labels and balances how many samples each client uses, to help a shared model generalize to new image domains. On three of four standard domain-generalization benchmarks, FedSB reports higher accuracy than prior federated domain generalization methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing hyperparameter-selection protocol threatens SOTA claim: reported ϵ/S may be tuned on test domains, so Table I gains may not survive a DomainBed-compliant selection.","rationale":"The paper proposes two simple components, label smoothing and a fixed per-client training budget, and reports state-of-the-art federated domain generalization accuracy on three of four standard benchmarks. The central claim is primarily empirical, so the most load-bearing concern is the validity of the comparison protocol rather than the mechanistic assumption behind the budget. The reader's weakest_assumption focuses on Eq. (8), which assumes that equalizing sample count equalizes influence; that concern is plausible but secondary because the ablation study suggests the budget helps even if the mechanism is imperfect. The missing hyperparameter-selection protocol is more serious: the paper's Section IV does not describe how ϵ and S were chosen, and Table VI demonstrates that these values materially affect accuracy. With a margin of only 1.45 points over the strongest baseline on PACS, the reported advantage could be an artifact of peeking at test-domain results. Because the code is public, this concern can be resolved quickly; therefore a conditional verdict remains appropriate. I would not move to reject without first checking the repository. I partially agree with the reader because the reader's rationale also flags missing protocol details, although the formal weakest_assumption field identifies Eq. (8) instead.","tokens_in":8261,"tokens_out":8225,"duration_ms":75189,"concrete_test":"Download the public FedSB repository and re-run the PACS leave-one-domain-out experiments using the official DomainBed selection protocol: create a validation split inside each source domain, select ϵ and S based only on that validation accuracy, then evaluate on the held-out target domain. Compare the average to the reported 83.81 and to the FedIIR baseline run under the same protocol. If the average drops below FedIIR's, or if the code contains per-dataset ϵ/S values that were not selected by such a protocol, the SOTA claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline SOTA claim (Table I, three of four datasets) rests on two hyperparameters: the smoothing coefficient ϵ and the per-client budget S. Section IV (Implementation details) fixes the optimizer, learning rate, and batch size, but never states how ϵ and S are selected for each dataset. Table VI shows that changing these values moves PACS accuracy by up to 3 points (e.g., S=45B gives 80.27 vs the reported 83.81 for the full model). Since FedSB beats FedIIR on PACS by only 1.45 points (83.81 vs 82.36), test-set-guided selection of ϵ/S could fully account for the claimed improvement. The paper describes no validation split or DomainBed-compliant model-selection rule; without that information, the empirical advantage is not independently verifiable. This is more load-bearing than the sample-count assumption in Eq. (8): even if the balancing mechanism is imperfect, the method could still improve accuracy, but if the comparison protocol is biased, the central claim collapses.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes FedSB, a federated domain generalization method that combines client-level label smoothing with a per-client training budget. Label smoothing replaces hard one-hot labels with soft labels to reduce client overconfidence, while the budget mechanism resamples every local dataset to a fixed size S by under-sampling large clients and oversampling small clients. The authors argue, via Eq. (8), that this ensures equal contribution of clients to the global update. They evaluate FedSB on PACS, OfficeHome, TerraIncognita, and VLCS under the leave-one-domain-out protocol, reporting state-of-the-art accuracy on three of the four datasets, and include ablations on PACS showing that both components improve accuracy over FedAvg. Additional experiments use ResNet-50 and Vision Transformer backbones, and the code is publicly released.","tokens_in":8439,"tokens_out":6565,"duration_ms":58273,"significance":"If the empirical claims hold, FedSB is a simple and inexpensive addition to federated domain generalization: it requires no extra communication, no style sharing, and no auxiliary networks, only a resampling step and a modified loss. The manuscript is clearly written and gives credit-worthy practical details: ablations isolate the contribution of each component, sensitivity analysis is provided, multiple backbones are tested, and the code is released. The central risk is not the internal consistency of the derivation but the external validity of the benchmark numbers: the hyperparameter selection protocol is not described, and the reported margins over the strongest baseline are small enough that test-set-guided selection of epsilon and S could account for them. The contribution is therefore promising but not yet fully substantiated.","major_comments":[{"comment":"The manuscript never states how the label-smoothing coefficient epsilon and the per-client budget S are chosen, nor does it report the final values used for each dataset in Table I. Table VI shows that varying S alone changes PACS average accuracy from 80.27 to 81.51, and varying epsilon changes it from 82.98 to 83.35, while FedSB's reported margin over FedIIR on PACS is only 1.45 points. Without a stated model-selection rule (e.g., the training-domain validation split used in the DomainBed protocol), the claimed state-of-the-art results could be the product of test-domain-guided hyperparameter tuning. Please specify the selection protocol and report the per-dataset values of epsilon and S, including the meaning of the 'B' unit in entries such as S=45B.","section":"IV (Implementation details), Tables I and VI"},{"comment":"Equation (8) shows that after resampling every local dataset to S samples, the expected global update is an average over K clients of per-client gradient sums, with a common factor S/(KB). This establishes equal sample counts, but not equal influence on the learned model: random subsampling of large clients may alter class balance, and oversampling small clients duplicates samples and can amplify label noise. The paper does not analyze these effects, so the statement that the budget 'ensures equal contribution from all clients' is stronger than what the derivation supports. The ablation in Table V indicates an empirical benefit on PACS, but the mechanistic claim needs qualification or additional analysis.","section":"III, Eq. (8)"},{"comment":"FedADG is the only baseline not re-run by the authors; its numbers in Table I are taken from the original paper. If FedADG was trained with a different backbone, learning-rate schedule, or number of communication rounds, the comparison in Table I is not apples-to-apples. The authors should either re-run FedADG under the same protocol or provide a clear statement of the differences and justify that the comparison remains fair.","section":"IV, Baselines paragraph"}],"minor_comments":[{"comment":"The header 'FebSB' in Tables II and III should be 'FedSB'.","section":"Tables II and III"},{"comment":"The numbers in Table I are concatenated without spacing (e.g., '91.67±0.979.25±1.3...'), making the table difficult to parse; please format the entries with proper separators.","section":"Table I"},{"comment":"The phrase 'were ˆL denotes' should be 'where ˆL denotes'.","section":"III, after Eq. (8)"},{"comment":"The text defines S as a number of samples, but Table VI uses units such as '30B' and '45B', suggesting batches; please clarify the relation between S and the batch size used in training.","section":"III, Section III and Table VI"},{"comment":"The dataset name is written as 'TerraINC' in Table I and as 'TerraIncognita' elsewhere; please standardize the notation.","section":"Throughout"},{"comment":"The paper says 'Following [11], we utilize the leave-one-domain-out setting' but does not mention whether the DomainBed training-domain validation rule for model selection is followed; even if a different rule is used, it should be stated explicitly.","section":"IV, Implementation details"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and the empirical evidence is suggestive, but the missing hyperparameter-selection protocol is the make-or-break point. If the authors can demonstrate that their results survive a DomainBed-compliant or otherwise clearly specified selection rule, the paper is likely acceptable. I would not reject at this stage, but the current manuscript does not yet allow independent verification of the central SOTA claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [name],\n\nFedSB is a straightforward combination of two known ideas—client-level label smoothing and a fixed per-client sample budget—applied to federated domain generalization. The experiments are plausible and the ablation shows both components matter. But the paper never describes how ε and S are chosen, and the reported gains over FedIIR are small (1.45 points on PACS). If those hyperparameters were tuned on the test domains, the SOTA claim evaporates. That is the main thing to know.\n\nWhat is actually new is the combination and its evaluation on four DomainBed datasets, not the ingredients. The math in Eqs. (4)–(8) is standard expectation manipulation; Eq. (8) is true by construction once every client trains on S samples, and it is not used to back out accuracy numbers. No circularity. The paper is honestly written, gives ablations, sensitivity tables, and a public code link.\n\nThe soft spot is the protocol. Section IV fixes optimizer, learning rate, and batch size, but is silent on ε and S. Table VI shows the PACS average moving from 80.27 (S=45B) to 83.81 (full model), while the margin over FedIIR is just 1.45 points. Test-set-guided selection could fully account for the difference. This is a load-bearing gap, more so than the reader's concern about oversampling small noisy clients—that one is secondary because the method could still help even if the balancing mechanism is imperfect. A second minor issue: Tables V and VI have no error bars, so the ablation differences may be noise. The 'novel budgeting' framing overstates what is simply under/oversampling.\n\nThe paper is for people in federated domain generalization who want a cheap, communication-free accuracy boost. It deserves a serious referee, but the referee should demand the hyperparameter selection rule and per-dataset ε/S values, plus error bars on ablations, before the SOTA claim is accepted.\n\nMy recommendation: engage with it, but require the protocol details.","headline":"FedSB is a plausible, simple FDG method whose SOTA claim depends on an undocumented hyperparameter selection protocol.","tokens_in":8981,"tokens_out":2386,"would_cite":false,"duration_ms":20187,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedSB applies client-level label smoothing and a fixed per-client training budget to federated domain generalization, reporting state-of-the-art accuracy on three of four standard domain-shift benchmarks.","keywords":["federated learning","domain generalization","federated domain generalization","label smoothing","data heterogeneity","client budgeting","decentralized training","domain shift"],"falsifier":"Construct a two-client version of PACS in which the small-client domain has deliberately noisy labels while the large-client domain is clean, and compare FedSB under budgets $S$ equal to the small and large dataset sizes. If the fixed-budget rule in Eq. (8) is correct, equalizing the noisy small client should not hurt; observing a large accuracy drop when the noisy small client is force-equalized would falsify the equal-contribution premise.","tokens_in":8063,"feed_emoji":"🌐","tokens_out":7048,"duration_ms":61143,"temperature":0.7,"pith_summary":"FedSB is a federated learning method for domain generalization built from two ordinary-looking changes to standard federated averaging. Each client trains with label-smoothed targets, replacing hard one-hot labels with a mixture that keeps most probability on the true class and spreads the rest uniformly, which the authors argue reduces local overconfidence and prevents overfitting to domain-specific features. In addition, every client is forced to train on the same number of samples per round by randomly subsampling large local datasets and oversampling small ones, so no client dominates the global model merely because it has more data. The paper reports that this combination achieves state-of-the-art average accuracy on three of the four DomainBed benchmarks used (PACS, OfficeHome, and TerraIncognita), with an ablation on PACS showing each component helps. If the result holds, two simple and privacy-preserving tweaks capture much of the benefit that more elaborate federated domain generalization methods buy with extra communication or shared statistics.","feed_headline":"Soft labels and equal client budgets win on 3 of 4 domain-shift tests","feed_subtitle":"Federated learning with label smoothing and fixed per-client training budgets beats prior methods on three of four domain-shift benchmarks.","key_machinery":"The load-bearing object is the pair formed by the client-level label-smoothing loss and the fixed training budget $S$. Label smoothing replaces the one-hot target with $y'_c = 1-\\epsilon+\\epsilon/M$ for the true class and $\\epsilon/M$ elsewhere; expanded, this makes the loss a weighted sum of the negative log-likelihood on the true class and a smooth loss that penalizes confident wrong-class probabilities. The budget mechanism resamples each local dataset to exactly $S$ samples before local training, which in Eqs. (7)--(8) removes the $|\\mathcal{D}_i|$ factor from the expected global update. What the pair does is carry the causal story: local overconfidence and dataset-size imbalance are two separate, addressable mechanisms that hurt the aggregated model, and each can be fixed without changing the federated communication protocol.","core_discovery":"FedSB's central claim is that overconfident local models and unequal client data volumes are the two main correctable causes of poor cross-domain generalization in federated learning. Label smoothing changes the local loss to $(1-\\epsilon)(-\\log p_y) + \\epsilon(-\\frac{1}{M}\\sum_c \\log p_c)$, so a client is rewarded for calibrated, less certain predictions rather than for locking onto its own domain's shortcuts. The budgeting mechanism resamples each local dataset $\\mathcal{D}_i$ to a fixed size $S$, and Eq. (8) then makes the expected global update equal to $\\Theta^t - \\frac{\\eta S}{KB}\\sum_i \\mathbb{E}[\\nabla \\hat{\\mathcal{L}}_i]$, independent of $|\\mathcal{D}_i|$. With both components, the paper reports averages of 83.81 on PACS, 70.92 on OfficeHome, 46.89 on TerraIncognita, and 76.64 on VLCS, and its ablation on PACS shows that removing either the smoothing or the budget lowers the average accuracy.","pith_inferences":["Because the budget step only resamples data and does not alter the aggregation rule, FedSB's balancing idea should combine with FedProx-style proximal terms or prototype-sharing methods, but the paper does not test those combinations.","A natural check the paper does not run is to make the largest client the cleanest and easiest domain; if equalizing budgets still helps, the benefit is not simply removing a dominant domain, and if it hurts, the equal-contribution premise needs qualification.","The label-smoothing component could in principle be applied to client-level objectives in non-federated domain generalization as well, where the same overconfidence problem appears, but that transfer is outside the paper's experiments."],"forward_implications":["Client-level label smoothing can be dropped into any federated averaging scheme without sharing data or changing communication, so it is a cheap plug-in for improving unseen-domain accuracy.","Equalizing training budgets removes the dataset-size term from the expected global update, so sample-rich clients no longer steer the model toward their own domain.","The reported state-of-the-art averages on PACS (83.81), OfficeHome (70.92), and TerraIncognita (46.89) suggest the method transfers across very different domain-shift types.","Ablations on PACS indicate both ingredients matter: smoothing alone gives 83.31, budgeting alone gives 81.51, and the full method gives 83.81.","The method also improves over FedAvg with ViT backbones (for example, PACS ViT-b/16 average 88.56 versus 87.24), so the benefit is not tied to ResNet architectures."],"supporting_citations":[{"why":"Supplies the FedAvg aggregation baseline that FedSB modifies and compares against.","marker":"[1]"},{"why":"FedSR is the prior federated domain generalization regularizer baseline that FedSB competes with on all four benchmarks.","marker":"[6]"},{"why":"FedProx is a heterogeneity-focused federated baseline used in the main comparison table.","marker":"[10]"},{"why":"Defines the leave-one-domain-out DomainBed evaluation protocol used for all experiments.","marker":"[11]"},{"why":"FedIIR is the strongest recent FDG baseline in Table I, which FedSB outperforms on three datasets.","marker":"[25]"},{"why":"Supplies the PACS benchmark where FedSB reports its largest state-of-the-art gain.","marker":"[12]"},{"why":"Supplies the OfficeHome benchmark used for the second state-of-the-art result.","marker":"[13]"},{"why":"Supplies the TerraIncognita benchmark used for the third state-of-the-art result.","marker":"[14]"},{"why":"Supplies the VLCS benchmark, the one dataset where FedSB does not surpass FedIIR.","marker":"[15]"}],"fun_headline_variants":["Soft labels and fair client budgets boost federated generalization","Federated learning with label smoothing and balanced budgets wins 3/4","Balanced training and soft labels improve cross-domain federated models","Smoothing and equal client loads beat prior methods in domain shift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The balancing mechanism assumes that sample count, rather than domain difficulty, class imbalance, or label noise, is the main source of client influence, so that randomly shrinking large clients and oversampling small clients to the same size $S$ equalizes contributions without introducing harmful distortion.","fun_headline_variants_meta":{"raw":{"variants":["Soft labels and fair client budgets boost federated generalization","Federated learning with label smoothing and balanced budgets wins 3/4","Balanced training and soft labels improve cross-domain federated models","Smoothing and equal client loads beat prior methods in domain shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1461,"prompt_tokens":914,"completion_tokens":547,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":474}},"tokens_in":530,"tokens_out":547,"duration_ms":5295,"temperature":1.0,"reasoning_tokens":474,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:57:36.439914+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a two-client version of PACS in which the small-client domain has deliberately noisy labels while the large-client domain is clean, and compare FedSB under budgets $S$ equal to the small and large dataset sizes. If the fixed-budget rule in Eq. (8) is correct, equalizing the noisy small client should not hurt; observing a large accuracy drop when the noisy small client is force-equalized would falsify the equal-contribution premise.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Supplies the FedAvg aggregation baseline that FedSB modifies and compares against."},{"cited_title":"Fedsr: A simple and effective domain generalization method for federated learning,","cited_arxiv_id":null,"evidence_quote":"FedSR is the prior federated domain generalization regularizer baseline that FedSB competes with on all four benchmarks."},{"cited_title":"In search of lost domain generalization,","cited_arxiv_id":null,"evidence_quote":"Defines the leave-one-domain-out DomainBed evaluation protocol used for all experiments."},{"cited_title":"Out-of-distribution generalization of federated learning via implicit invariant relationships,","cited_arxiv_id":null,"evidence_quote":"FedIIR is the strongest recent FDG baseline in Table I, which FedSB outperforms on three datasets."},{"cited_title":"Deeper, broader and artier domain generalization,","cited_arxiv_id":null,"evidence_quote":"Supplies the PACS benchmark where FedSB reports its largest state-of-the-art gain."},{"cited_title":"Recognition in terra incognita,","cited_arxiv_id":null,"evidence_quote":"Supplies the TerraIncognita benchmark used for the third state-of-the-art result."},{"cited_title":"Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,","cited_arxiv_id":null,"evidence_quote":"Supplies the VLCS benchmark, the one dataset where FedSB does not surpass FedIIR."}],"review_version":1}