{"id":"3ee25da8-29c6-4ebb-8121-3febe6ae3d89","arxiv_id":"2412.10897","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"pFed-Mul integrates multi-output Gaussian processes with Pólya-Gamma augmented variational inference to perform mixed classification and regression in federated learning.","lead":"A new federated learning method, pFed-Mul, lets each local device jointly train a classification task and a regression task at the same time using multi-output Gaussian processes, while a central server shares a global prior. It targets uncertainty estimation and few-shot performance, but the paper's own tables show its regression results are sometimes worse than simpler baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The deterministic inducing-point conditional in Appendix C Eq. (13) is an unvalidated DTC projection that can manufacture the claimed calibration and OOD gains.","rationale":"The reader's weakest assumption already identifies the deterministic inducing-point approximation in Eq. (13) as a key risk, and my stress-test sharpens that concern: this is the DTC approximation, a known source of overconfident predictive variance in sparse GP literature. Since the paper's headline experiments include calibration and OOD detection, the approximation is load-bearing for the central claim, not merely a numerical detail. However, the concern is empirical rather than a proven mathematical contradiction: the method is coherent, the code is available, and the derivations are detailed enough to implement. A targeted experiment that removes the approximation by setting M to the training-set size would settle whether the reported uncertainty gains survive. I therefore keep the reader's CONDITIONAL verdict rather than escalating to REJECT. I do not attack the authors' conduct; the issue is that an unvalidated approximation sits directly under the strongest uncertainty claims. The Table 1 result that pFed-Mul is sometimes worse than pFed-St on CelebA regression is a secondary consistency problem, but it does not replace the DTC concern as the most load-bearing issue for the central claim.","tokens_in":21754,"tokens_out":5923,"duration_ms":60117,"concrete_test":"Re-run the 50-shot CelebA setting with the same protocol but set the number of inducing points M equal to each client's training-set size, so that Eq. (13) becomes exact, and compare accuracy, MSE, ECE, and OOD predictive-variance ratios against the reported M << N runs. If ECE or OOD separation changes by more than a small tolerance (e.g., ΔECE > 0.02) or accuracy moves materially, the DTC projection in Eq. (13) is responsible for the claimed gains; if all metrics are essentially unchanged, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix C Eq. (13) replaces the conditional distribution p(f_{x_n} | f_{x_m}) by its mean, so the variational q(f_{x_m}) in Eq. (14) is fit to a likelihood in which each training latent is a deterministic linear function of the inducing outputs. This is the DTC / projected-process approximation, which is known to underestimate posterior variance and overestimate the marginal likelihood when M << N. The paper's central claims of calibrated uncertainty and superior OOD detection in Section 5.3 are exactly the quantities this approximation can corrupt, yet no comparison is made against an exact MOGP posterior, a fully variational sparse GP, or MCMC. The 2-iteration local VI is secondary; even with fully converged VI, Eq. (13) imposes a point-mass conditional that is not part of the Bayesian model and carries no KL penalty. Consequently, the reported ECE = 0.054 and the OOD variance spikes could be properties of the projection rather than of the underlying MOGP model.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes pFed-Mul, a personalized Bayesian federated learning method in which each client fits a multi-output Gaussian process (MOGP) over correlated classification and regression tasks, using Pólya-Gamma augmentation and mean-field variational inference to obtain closed-form local posterior updates. The server aggregates local ELBOs to update the global MOGP prior hyperparameters, including the deep-kernel backbone, and redistributes the updated prior to clients. Experiments on synthetic data, CelebA, and Dogcat claim superior predictive accuracy, calibration, OOD detection, and convergence rate. The main technical contribution is the analytical local inference scheme and a server-side ELBO optimization for the global prior.","tokens_in":21948,"tokens_out":7252,"duration_ms":67003,"significance":"If the claims held, the paper would make a useful step: a Bayesian FL method that handles task diversity with closed-form updates and calibrated uncertainty is valuable, and the variational derivation in Appendices A-D is internally consistent and standard. The public code, explicit analytical expressions, and the use of Pólya-Gamma augmentation to enable mean-field VI are strengths. However, the current experimental evidence does not support several of the paper's central claims, and the sparse-GP approximation used for the uncertainty quantities is not validated. These issues are substantive but fixable within the manuscript's scope.","major_comments":[{"comment":"The statement that 'pFed-Mul consistently outperforms existing methods across almost all scenarios' is directly contradicted by the reported regression results on CelebA. For the 10-shot/20-client, 20-shot/15-client, and 50-shot/10-client settings, pFed-Mul has MSE 0.488, 0.476, and 0.301, whereas FedPer has 0.369, 0.328, and 0.261, and the single-task variant pFed-St has 0.690, 0.321, and 0.221. Thus pFed-Mul is worse than both baselines in all three settings, and the multi-task framework does not improve regression over the single-task variant on this dataset. The abstract and the 'most significant improvements' sentence need to be revised, or the result needs to be explained and supported by additional experiments.","section":"Section 5.2.2, Table 1"},{"comment":"Replacing the conditional distribution p(f_{x_n} | f_{x_m}) with its conditional mean is the deterministic training conditional (DTC / projected-process) approximation. This projection removes the K_{xx} - Q_{xx} term from the predictive variance and is known to underestimate posterior uncertainty when the number of inducing points M is much smaller than N. The calibration and OOD results in Section 5.3 (ECE 0.054 and the variance spikes in Figure 4) are precisely the quantities this approximation can corrupt, and no comparison is made against an exact MOGP posterior, a fully variational sparse GP, or MCMC. In addition, the experimental protocol does not state the number of inducing points M or whether the sparse approximation in Eq. (13) is active in the reported results. Please add an approximation-error study or substantially soften the uncertainty claims.","section":"Appendix C, Eq. (13)"},{"comment":"The claimed 'superior' OOD detection is not supported by the reported evidence. The only result is a visual inspection of predictive variance under two masked images for pFed-Mul alone; no quantitative OOD metric (e.g., AUROC or similar) and no baseline comparison are provided. Section 5.3 states that the method achieves superior performance to previous baselines in OOD detection, but no baseline OOD numbers appear anywhere. Please add a quantitative OOD evaluation or revise the claim.","section":"Section 5.3.2, Figure 4"},{"comment":"The convergence-rate claim rests on a single trajectory over the first 10 communication rounds, with no repetitions, error bars, or significance analysis. Since all methods are run with two local updates per round, the plot does not show whether pFed-Mul's advantage comes from the Pólya-Gamma natural-gradient property or from initialization and hyperparameter choices. Please report means and standard deviations over multiple seeds and, ideally, a matched-compute comparison.","section":"Section 5.4, Figure 5"}],"minor_comments":[{"comment":"For the synthetic experiment, all hyperparameters are initialized as the ground truth; this makes the result a sanity check of the inference machinery rather than evidence that the global prior is learned from data. Please state this interpretation explicitly.","section":"Section 5.1.1"},{"comment":"No standard deviations or numbers of seeds are reported in Table 1, and Figure 5's legend lists 'PerFed' while the tables use 'pFedMe'; please fix the label and report variability.","section":"Table 1 and Figure 5"},{"comment":"The ECE reliability analysis is reported for CelebA only; adding the Dogcat calibration result would strengthen the comparison.","section":"Section 5.3.1"},{"comment":"The notation x_n is used ambiguously: Eq. (13) calls x_n predictive points, while Eq. (14) uses the same notation in likelihood terms for training data; please define the index sets explicitly.","section":"Appendix C"},{"comment":"The regression labels are constructed by adding Gaussian noise to the classification labels, so the two tasks are nearly deterministic functions of each other; this is an artificial correlation that may overstate the benefit of MOGP transfer on this benchmark.","section":"Section 5.1.1, Dogcat"},{"comment":"The ELBO optimization in Eq. (19) uses AdamW for kernel hyperparameters and mixing weights, while Eq. (20) gives a closed form for the noise variances; the paper does not report the learning rate or optimizer hyperparameters, which limits reproducibility.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The paper is within KDD's scope and the derivations are competent, but the experimental validation currently lags behind the strength of the claims. I would require the Table 1 contradiction to be resolved, the sparse-GP approximation to be validated, and the OOD/convergence claims to be supported by quantitative evidence before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is the integration of multi-output Gaussian processes with Pólya-Gamma augmented mean-field VI inside a federated loop, so each client can handle both classification and regression with uncertainty. That is a legitimate extension, and the derivations in the main text and appendices are coherent. The analytical local updates are a real convenience, and the code is public, so the method is implementable.\n\nWhat the paper does well: the ELBO derivation is standard but clean; the server-side aggregation of posteriors is sensible; the synthetic experiment, while small, shows the multi-task transfer mechanism working; and the classification accuracy on CelebA is consistently the best or near-best. The ablation studies are also reasonable.\n\nThe soft spots are mostly in the evidence, not the math. The abstract and Section 5.2.2 claim consistent superiority, but Table 1 shows pFed-Mul's regression MSE is worse than FedPer and pFed-St on all three CelebA settings. The synthetic setup initializes hyperparameters at ground truth, so it is a sanity check rather than a validation. There are no error bars anywhere.\n\nThe more serious issue is the inducing-point approximation. Appendix C Eq. (13) replaces the conditional distribution p(f_data | f_inducing) with its mean—the DTC / projected-process approximation. That is known to underestimate posterior variance. The paper's headline uncertainty results—ECE of 0.054 and the OOD variance spikes—are exactly the quantities this approximation can corrupt. There is no comparison against MCMC, a converged VI solution, or a fully variational sparse GP. So the calibration and OOD claims are suggestive, but not established; the stress-test note lands.\n\nI would not call this fatal. The predictive accuracy story is plausible, the model is well specified, and the method is new enough to be worth engagement. But the paper needs a small-scale validation of the DTC projection, error bars, and a more honest summary of Table 1 before the uncertainty claims can be taken at face value.\n\nIf this came in as a new submission, I'd send it to review with those requests. It is a solid subfield contribution that needs strengthening, not a desk reject.","headline":"A clean combination of Pólya-Gamma augmentation with multi-output GPs for mixed-task federated learning, but the uncertainty claims outrun the evidence, mainly because the sparse approximation is an unvalidated DTC projection.","tokens_in":22490,"tokens_out":2615,"would_cite":true,"duration_ms":26972,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that federated learning can handle correlated classification and regression tasks on the same device by coupling them through a multi-output Gaussian process, and that the resulting Bayesian method outperforms…","keywords":["Bayesian federated learning","multi-task learning","multi-output Gaussian process","Pólya-Gamma augmentation","mean-field variational inference","uncertainty calibration","out-of-distribution detection","personalized federated learning"],"falsifier":"Run pFed-Mul on the same few-shot CelebA and Dogcat splits while increasing the number of mean-field iterations and local updates, or replacing the deterministic inducing-point projection with a proper variational bound, and compare predictive accuracy, ECE, and OOD variance; if any of these change materially, the cheap approximation is responsible for the claims.","tokens_in":21522,"feed_emoji":"🤖","tokens_out":7600,"duration_ms":65170,"temperature":0.7,"pith_summary":"Federated learning usually assumes every device is solving the same kind of task, but real devices often collect both categorical labels and continuous targets at once. This paper proposes pFed-Mul, a Bayesian federated method where each client models correlated classification and regression tasks jointly through a multi-output Gaussian process, and the server updates a global prior by averaging the clients' variational posteriors. The paper claims that, in few-shot settings on synthetic data, CelebA, and Dogcat, this joint modeling improves predictive accuracy, out-of-distribution detection, uncertainty calibration, and convergence rate compared with single-task federated baselines. The technical core is making classification inference tractable: Pólya-Gamma augmentation turns the non-conjugate logistic likelihood into a conditionally conjugate one, so local updates are closed-form mean-field iterations. A sympathetic reader would care because this is a concrete route to serving heterogeneous real-world clients with limited data while still getting calibrated uncertainty.","feed_headline":"Mixed-task federated learning beats single-task baselines","feed_subtitle":"A multi-output Gaussian process with Pólya-Gamma inference gives better few-shot accuracy and calibrated uncertainty on clients.","key_machinery":"The machinery is the pair formed by a multi-output Gaussian process prior and Pólya-Gamma augmentation. The MOGP prior, built from $B$ independent basis Gaussian processes with a mixing weight matrix $\\mathbf{W}$, defines a cross-covariance between every pair of classification and regression latent functions, which is what lets one task type borrow strength from the other. Pólya-Gamma augmentation introduces auxiliary variables $\\boldsymbol{\\omega}$ that make the logistic classification likelihood Gaussian in the latent functions given $\\boldsymbol{\\omega}$, so the joint model becomes conditionally conjugate. Mean-field variational inference then alternates between closed-form updates: $q_1(\\boldsymbol{\\omega})$ is a Pólya-Gamma distribution and $q_2(\\mathbf{f}) = \\mathcal{N}(\\mathbf{m}, \\boldsymbol{\\Sigma})$ with $\\boldsymbol{\\Sigma} = (\\mathbf{H} + \\mathbf{K}^{-1})^{-1}$ and $\\mathbf{m} = \\boldsymbol{\\Sigma}\\mathbf{H}\\mathbf{v}$. This closed-form structure is what lets the server aggregate posteriors and maximize an averaged ELBO analytically, and it is also what the paper credits for fast convergence, via the equivalence between mean-field VI on a conditionally conjugate model and natural-gradient descent with unit step size.","core_discovery":"The central claim, stated on the paper's own terms, is that task diversity on federated clients should be modeled, not ignored. pFed-Mul assumes each client has $T_r$ regression and $T_c$ classification tasks whose latent functions are drawn from a multi-output Gaussian process with a linear model of coregionalization, so correlated tasks share statistical strength. On the server, the averaged evidence lower bound over clients is maximized to update the global MOGP prior, which is then redistributed as the next round's prior. The paper reports that pFed-Mul consistently outperforms existing methods across almost all of its few-shot scenarios, with the largest gains of $0.155$ in regression mean squared error and $3.86\\%$ in classification accuracy, the best expected calibration error among baselines, clearly elevated predictive variance on OOD images, and faster early-round convergence.","pith_inferences":["Editorial extension: the Pólya-Gamma route the paper develops for binary classification can in principle be carried to multi-class classification through logistic-softmax or one-vs-each softmax approximations, which the paper mentions in an appendix but does not implement; testing that extension is a direct next step.","Editorial extension: because the empirical claims rest on only two mean-field iterations and two local updates per round, a natural stress test is to increase those counts and check whether accuracy and calibration stay stable; the paper does not report such a sensitivity curve.","Editorial extension: the deterministic inducing-point projection in Appendix C replaces a conditional distribution by its mean, so the reported OOD variance could understate posterior uncertainty; comparing against a full variational treatment of the inducing points would indicate how much of the calibration gain is due to the model versus the approximation.","Editorial extension: the framework treats task diversity as a resource for knowledge transfer rather than merely a source of heterogeneity, suggesting that future federated systems could actively schedule which task types a device contributes."],"forward_implications":["If pFed-Mul is correct, a federated system can train a device's correlated classification and regression tasks simultaneously, improving both relative to separate single-task models, especially with few-shot data.","Predictive variance from the MOGP posterior is usable as an OOD signal: masked or semantically shifted inputs show higher variance in the reported experiments.","Uncertainty calibration improves: pFed-Mul records the lowest expected calibration error among the compared methods on the CelebA classification task.","Convergence in the first ten communication rounds is faster and more stable than the compared baselines, with only two mean-field iterations and two local updates per round.","The global MOGP prior learned by the server can be distributed to new clients, giving them a personalized posterior after local inference with their own data."],"supporting_citations":[{"why":"Supplies the Pólya-Gamma augmentation representation of the logistic likelihood that makes local inference conditionally conjugate.","marker":"[43]"},{"why":"The personalized GP-based federated baseline whose posterior-aggregation design pFed-Mul extends to mixed task types.","marker":"[1]"},{"why":"Defines the multi-output Gaussian process and linear model of coregionalization used to couple classification and regression tasks.","marker":"[3]"},{"why":"Establishes the server-side aggregation of local posteriors into a global prior that pFed-Mul adapts from the parametric to the nonparametric setting.","marker":"[67]"},{"why":"Provides the result that mean-field VI on a conditionally conjugate model equals natural-gradient descent with step size one, used to explain fast convergence.","marker":"[23]"},{"why":"Supplies the variational inducing-points framework that reduces local complexity to linear in the number of samples.","marker":"[56]"},{"why":"Supports the use of shared inducing points across heterogeneous outputs in the MOGP.","marker":"[40]"},{"why":"Introduces deep kernels, used to give the MOGP a flexible input transformation in the image experiments.","marker":"[61]"}],"fun_headline_variants":["Bayesian FL unifies classification and regression","Task-diverse federated learning beats single-task","Multi-task federated learning with Gaussian processes","Pólya-Gamma inference empowers federated multi-task learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that two mean-field iterations and two local updates per round, together with the deterministic inducing-point projection, produce a posterior that is faithful enough that the reported accuracy, calibration, and OOD gains reflect the model rather than artifacts of the approximation.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian FL unifies classification and regression","Task-diverse federated learning beats single-task","Multi-task federated learning with Gaussian processes","Pólya-Gamma inference empowers federated multi-task learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2565,"prompt_tokens":895,"completion_tokens":1670,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":1610}},"tokens_in":511,"tokens_out":1670,"duration_ms":12789,"temperature":1.0,"reasoning_tokens":1610,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:30:15.992147+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run pFed-Mul on the same few-shot CelebA and Dogcat splits while increasing the number of mean-field iterations and local updates, or replacing the deterministic inducing-point projection with a proper variational bound, and compare predictive accuracy, ECE, and OOD variance; if any of these change materially, the cheap approximation is responsible for the claims.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Pólya-Gamma augmentation representation of the logistic likelihood that makes local inference conditionally conjugate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The personalized GP-based federated baseline whose posterior-aggregation design pFed-Mul extends to mixed task types."},{"cited_title":"Álvarez, Lorenzo Rosasco, and Neil D","cited_arxiv_id":null,"evidence_quote":"Defines the multi-output Gaussian process and linear model of coregionalization used to couple classification and regression tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the server-side aggregation of local posteriors into a global prior that pFed-Mul adapts from the parametric to the nonparametric setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the result that mean-field VI on a conditionally conjugate model equals natural-gradient descent with step size one, used to explain fast convergence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the variational inducing-points framework that reduces local complexity to linear in the number of samples."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the use of shared inducing points across heterogeneous outputs in the MOGP."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces deep kernels, used to give the MOGP a flexible input transformation in the image experiments."}],"review_version":1}