Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions

Christoph H. Lampert; Hossein Zakerinia; Jonathan Scott

REVIEW 2 major objections 7 minor 15 references

Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions

T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that personalized federated models can be generated for clients with no labeled data, by training a hypernetwork that maps a client's unlabeled examples to low-dimensional subspace parameters.

desk verdict Solid empirical method, garbled proof in the appendix that needs fixing before the theory claim is taken seriously. read the letter →

arxiv 2505.15579 v1 pith:SC4FSYSE submitted 2025-05-21 cs.LG stat.ML

Hossein Zakerinia , Jonathan Scott , Christoph H. Lampert This is my paper · ORCID

classification cs.LGstat.ML

keywords personalizedfederatedlearningunlabeledclientshypernetworkslow-dimensionalsubspacePAC-Bayesgeneralizationboundtransductivemulti-taskstatisticalheterogeneity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FLowDUP aims to show that a federated-learning client with no labels at all can still receive a personalized predictive model. The method trains a hypernetwork that reads a batch of unlabeled client examples and outputs a low-dimensional parameter vector $v \in \mathbb{R}^k$; a fixed random expansion matrix $P$ turns that vector into the full model $\theta = \theta_0 + P v$. The training objective is derived from a new transductive multi-task PAC-Bayes bound, and it combines a loss computed on labeled clients with a regularizer that any client, labeled or not, can evaluate. If the claim holds, unlabeled clients obtain a model with a single forward pass, no local fine-tuning, and communication cost proportional to $k = 10^4$ rather than the full parameter count. Across class-partitioned CIFAR-10, rotated MNIST and Fashion-MNIST, and FEMNIST, the authors report that FLowDUP outperforms FedAvg, FedProx, LD-FedAvg, and FedTTA on unlabeled test clients, including when only 10% of training clients have labels.

What carries the argument

The load-bearing pieces are the hypernetwork and the subspace parameterization. The hypernetwork compresses an unlabeled dataset into a single averaged feature embedding $r(X) = (1/|X|)\sum_{x \in X} h_1(x)$, then maps that embedding through $h_2$ to $v \in \mathbb{R}^k$; the client-side classifier is $\theta = \theta_0 + P v$, where $P \in \mathbb{R}^{d \times k}$ is a fixed random expansion matrix. This is what makes personalization cheap: the server transmits only the hypernetwork and $\psi_r$, and the client never trains or fine-tunes a model. The theoretical engine is Theorem 4.1, a transductive multi-task PAC-Bayes bound that controls the risk of all clients using only labeled-client losses; its complexity terms are exactly the loss $L$ and regularizer $\Omega$ in the objective, which is why the paper claims the objective is theoretically motivated.

What would settle it

A controlled experiment would split clients into two groups whose inputs share the same marginal distribution but whose label assignments are opposite, for example by permuting class labels per client in the rotated-MNIST setup. If FLowDUP is trained with labels from both groups and then evaluated on unlabeled clients from each group, success requires that unlabeled clients recover their group's label mapping; a result near chance or near the population-average accuracy would falsify the claim that unlabeled marginals carry the personalization signal.

Watch

Extended reading notes

Core claim

The central claim is that the mapping from an unlabeled client dataset to a usable personalized classifier is learnable, and that it can be computed on-device in one forward pass. FLowDUP's generator is a hypernetwork $h(X) = h_2( (1/|X|) \sum_{x \in X} h_1(x) )$ that produces a low-dimensional subspace vector; the deployed classifier is $\theta = \theta_0 + P v$. The objective is justified by Theorem 4.1, a PAC-Bayes bound for transductive multi-task learning: it upper-bounds the average true risk of all $n$ clients, labeled and unlabeled, by the labeled training loss plus complexity terms, one of which is $\sum_i \|h(S_i; \psi_h) - \psi_r\|^2$ and is computable from unlabeled data alone. The bound's structure is what turns the regularizer into a trainable object, so unlabeled clients contribute to learning even though they contribute no loss. The authors' experiments position FLowDUP as the best-performing method on unlabeled test clients across four heterogeneous benchmarks, with ablations showing gains from including unlabeled clients in training and from learning the regularizer rather than fixing it at zero.

Load-bearing premise

The method's load-bearing assumption is that the distribution of a client's unlabeled inputs, summarized by the averaged features $r(X)$, contains enough information to determine the client-specific labeling rule; if two clients have the same input distribution but different correct labels, FLowDUP has no signal to separate them, as the authors note in Section 7.

Editorial extensions

If this is right

Any client that joins after training with no labels can receive the hypernetwork and generate a personalized model on-device with one forward pass.
Communication and compute scale with the subspace dimension $k = 10^4$, not the full parameter dimension—about 15× smaller for the CNN and 1100× smaller for ResNet18 in the reported setups.
Unlabeled clients are not just beneficiaries; their participation in training lowers the regularizer's complexity term and, per the ablations, improves accuracy most when labeled clients are scarce ($p=0.05$–$0.2$).
The transductive element of the bound improves the sample-complexity factor by $\sqrt{1 - n_L/n}$ relative to inductive bounds, so the theoretical guarantee is strongest exactly when most clients are unlabeled.
Because the expansion matrix and initialization are fixed and seed-shared, the server never needs to transmit full model weights, keeping the scheme compatible with the federated principle that local data stays on the device.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors leave implicit that the same hypernetwork-plus-subspace generator could serve as a lightweight personalization head for large foundation models, with the low-dimensional vector indexing a fixed adapter space instead of the full parameter space.
A testable extension would probe whether the averaged embedding $r(X)$ stays sufficient when clients share heavily overlapping input distributions but diverge only in labeling; the bound would still hold, but the empirical gap over non-personalized baselines might shrink.
The stated limitation suggests a practical safeguard the paper does not develop: combine FLowDUP with a small number of labeled examples or client metadata for high-stakes clients, since unlabeled-only personalization has an information-theoretic ceiling when conditionals differ without visible marginal differences.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Let me give you the short version: this is a genuinely useful empirical paper whose advertised theory has a real hole. FLowDUP is worth knowing about, but the proof in Appendix A is not, as written, a proof.

What's actually new: prior hypernetwork approaches for unlabeled clients (Amosy et al.; Scott et al.) generated full-dimensional models on the server and couldn't use unlabeled clients during training. FLowDUP outputs a low-dimensional subspace parameterization, so the hypernetwork is small enough to run on-device, and unlabeled clients contribute to the regularizer during training. The experiments are clean: multiple datasets, label and feature heterogeneity, ablations on subspace dimension, architecture, and the effect of unlabeled clients. Results are solid, and the authors are honest about the main limitation (the marginal distribution must carry enough information about the task).

The soft spots, in order of severity. First, Theorem A.1's proof is garbled. Equation (27) changes the left-hand side from R(ρ)-eR(ρ) to eR(ρ)-bR(ρ) without justification, equation (29) repeats that, and equation (30) applies the transductive bound to state the final inequality directly, which would make the second complexity term redundant. The intended structure is likely recoverable — use (26) to bound R-eR, then apply Bégin et al.'s bound at the task level to bound eR-bR — but as it stands the theorem is not established. Since Theorem 4.1 is the stated motivation for the training objective, this needs a real correction, not a cosmetic one. Second, the theory novelty is overstated: Pentina and Lampert (2017) already treated multi-task learning with labeled and unlabeled tasks; the paper says earlier theory only handles all-labeled tasks, which is not accurate. The actual contribution — low-dim subspaces, on-device hypernetwork, learning the regularizer from unlabeled clients — should be positioned against that work. Third, the theory assumes loss in [0,1] while experiments use cross-entropy, and no code is released. Minor for this paper, but worth noting.

Who should read this: anyone working on personalized FL with unlabeled clients or transductive meta-learning. The empirical method deserves referee time. My recommendation: send it to review, but make the corrected proof and the related-work positioning conditions for acceptance.

Referee Report

2 major / 7 minor

Summary. The paper proposes FLowDUP, a personalized federated learning method that produces a personalized model for a client from unlabeled data alone via a forward pass through a hypernetwork. The hypernetwork maps an averaged feature embedding of the client's unlabeled inputs to low-dimensional parameters in a random subspace, then expands them to full model parameters. The training objective combines a supervised loss on labeled clients with a learnable regularizer computable from unlabeled data. The authors state a transductive multi-task PAC-Bayesian generalization bound (Theorem 4.1, general version Theorem A.1) intended to justify the objective, and they present experiments on CIFAR-10, rotated MNIST/Fashion-MNIST, and FEMNIST showing consistent gains over baselines that can also serve unlabeled clients, together with ablation studies. The authors explicitly acknowledge in Section 7 that the method relies on the marginal input distribution providing enough information about the client-specific labeling function and may struggle otherwise.

Significance. If the theoretical results are valid, the paper makes a useful contribution: it enables personalization for unlabeled clients at low communication and computation cost, and it allows unlabeled clients to contribute to training. The experimental evaluation is thorough, includes multiple baselines and ablations, and the Figure 1 embedding visualization supports the proposed mechanism. However, the proof of the main theorem as written does not compose, so the theoretical contribution is not yet established. The loss-boundedness assumption in the theorem is also not met by the experiments. For these reasons, the paper requires major revision before the claims can be accepted; the empirical results alone are promising but do not by themselves support the theoretical statements.

major comments (2)

[Appendix A, Equations (27), (29), (30)] The proof of Theorem A.1 does not establish the claimed bound. Equation (26) correctly gives a uniform bound on R(A) - eR(A) for all algorithms A; taking the expectation over A ~ rho yields a bound on R(rho) - eR(rho), not the bound on eR(rho) - bR(rho) displayed in (27) and repeated in (29). Equation (30) is introduced as applying the transductive bound (11) to the pair (eR(rho), bR(rho)), but it is written as R(rho) <= bR(rho) + ..., which on its face would already be the final theorem without the second complexity term. The two displayed inequalities therefore cannot be combined to yield (14). The intended structure, namely bounding R(rho) - eR(rho) via the multi-task PAC-Bayes argument and eR(rho) - bR(rho) via the transductive bound of Begin et al., is recoverable, but the proof must be rewritten and the left-hand sides of (27), (29), and (30) corrected before Theorem A.1 is considered proven.
[Theorem 4.1 and Section 5 experiments] The theorem is stated for any loss function l taking values in [0,1], but the experiments use the cross-entropy loss, which is unbounded. Since the training objective L is presented as derived from the bound, the paper should either extend the theory to a bounded surrogate loss (for example, a clipped or sigmoid-transformed version) or explicitly state that the bound only loosely motivates the objective and does not directly apply to the cross-entropy loss used in practice. As written, the theoretical guarantee does not cover the actual empirical setup.

minor comments (7)

[Abstract and Section 7] The Abstract's claim that FLowDUP is able to generate a personalized model for 'any client' is too strong given the limitation explicitly acknowledged in Section 7, where the authors note that the method will struggle if the marginals (D_i)_X do not reveal the conditional distributions (D_i)_{Y|X}. Please qualify the abstract accordingly.
[Equation (8) and Appendix A] The square-root expressions in Theorem 4.1 and in several equations in Appendix A have ambiguous scope because parentheses are missing; please add parentheses to clarify what is inside the square root and what is in the denominator.
[Theorem 4.1] The constants c1 and c2 are only described as 'logarithmic terms in n and nL'. Please provide explicit expressions, or at least state the exact dependence, so that the theorem is fully self-contained.
[Proof of Theorem 4.1] The derivation of Theorem 4.1 from Theorem A.1 is compressed to a single sentence. Since the Gaussian specialization involves the variance parameters alpha_h, alpha_theta, and alpha_r and the identification of the C(A,Q,P) term, it would be helpful to show the KL computations explicitly in the appendix.
[Related Work and Experiments] The related work identifies Amosy et al. (2024) and Scott et al. (2024) as the closest prior methods, but no experimental comparison with them is provided. A comparison would help substantiate the claims that FLowDUP improves on these methods in using unlabeled clients during training and in operating in a low-dimensional subspace.
[Algorithm 2, line 9] The algorithm uses the deterministic regularization vector psi_r, whereas the theory considers a distribution Q = N(psi_r; alpha_r I). Please clarify the correspondence between the deterministic algorithm and the stochastic PAC-Bayes quantities.
[Reproducibility] No mention is made of code release. For a methods paper such as this, providing code or at least stating its availability would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the PAC-Bayes bound and the training objective are not equivalent by construction, and the self-citations are not load-bearing.

full rationale

FLowDUP's objective (Eq. 4-6) is presented as a trainable proxy obtained by minimizing terms on the right-hand side of Theorem 4.1; the theorem is a PAC-Bayes bound proved in Appendix A for stochastic meta-posteriors, and the paper explicitly states that the practical algorithm replaces stochastic models with deterministic networks and treats the regularization strength as a hyperparameter. The bound is not equivalent to the objective by definition: it upper-bounds true risk using training risk plus KL and complexity terms, which is the standard direction of a generalization guarantee rather than a rearrangement of the objective's own definition. The subspace parametrization θ = θ0 + P v is an assumed design choice (from Li et al. 2018) and is not derived from the bound, so no self-definitional reduction occurs. The cited prior works by the same authors (Zakerinia et al. 2024, 2025; Scott et al. 2024) provide framework and empirical motivation, but the transductive proof and the comparisons against external baselines (FedAvg, FedProx, FedTTA) supply independent content; the self-citations are not invoked as uniqueness theorems and do not force the result. The authors' own limitation statement in Section 7 about insufficient marginal information is an honest boundary condition, not a circularity. An apparent inconsistency in the Appendix A proof (Eqs. 27, 29, 30 swap the roles of R, eR, and bR) is a correctness concern for the theorem, but it is not a circularity step under the criteria used here.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method's practical success rests on a small number of hand-chosen design parameters (subspace dimension, regularization strength, sampling rate) and on the domain assumption that unlabeled input features encode the heterogeneity that matters. The theory imports standard PAC-Bayes tools and one transductive bound; the main unproven step is lifting the latter to the task level.

free parameters (5)

Subspace dimension k = 10^4 in main experiments, varied as 200, 500, 2000, 10000
User-selected design parameter; Table 4 shows accuracy rises monotonically with k, so the low-dimensional claim is parameterized by this choice.
Regularization strength lambda = tuned per dataset, value not reported
Strength of the Omega term in objective (4); tuned on validation clients, so central results partly depend on this model-selected hyperparameter.
Labeled-client sampling rate alpha = 0.9 in main experiments
Server sampling ratio for labeled clients in Algorithm 1; controls how much labeled supervision each round sees.
PAC-Bayes variance scales alpha_h, alpha_theta, alpha_r = not specified in experiments
Fixed variances in Theorem 4.1 and its proof; if the bound were evaluated numerically these scales would need to be chosen and would affect tightness.
Hypernetwork embedding dimension (h1 output) = 256
Architecture choice controlling the expressiveness of client embeddings; not swept except through the architecture variants in Table 3.

assumptions (5)

standard math PAC-Bayes change-of-measure inequality and Hoeffding's lemma
Used in the proof of Theorem A.1 to convert exponential moment bounds into KL-based risk bounds, Appendix A, Eqs. (19)-(23).
domain assumption Begin et al. (2014) transductive PAC-Bayes bound holds at the meta-task level
The proof of Theorem A.1 applies Eq. (11), originally stated for labeled and unlabeled samples within one task, directly to n tasks with nL labeled tasks, without derivation, Appendix A before Eq. (30).
domain assumption Random low-dimensional subspaces contain high-accuracy models
The parametrization theta = theta0 + P v assumes k much smaller than d suffices; the paper cites Li et al. (2018) and Zakerinia et al. (2025), and Table 4 shows performance degrades as k shrinks.
domain assumption Client heterogeneity relevant for prediction is visible in the marginal input distribution
FLowDUP's forward pass sees only unlabeled X; the authors explicitly state in Section 7 that if conditional distributions differ while marginals are insufficient, the method will struggle.
domain assumption Loss is bounded in [0,1]
Theorem 4.1 requires a loss ell: Y x Y -> [0,1], but experiments use unmodified cross-entropy; the gap is not discussed in the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions." pith.science (2026). https://pith.science/paper/SC4FSYSE

@misc{pith2026250515579,
  author       = {Pith},
  title        = {Pith review of: Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SC4FSYSE}},
  note         = {Machine review of arXiv:2505.15579}
}

read the original abstract

Personalized federated learning has emerged as a popular approach to training on devices holding statistically heterogeneous data, known as clients. However, most existing approaches require a client to have labeled data for training or finetuning in order to obtain their own personalized model. In this paper we address this by proposing FLowDUP, a novel method that is able to generate a personalized model using only a forward pass with unlabeled data. The generated model parameters reside in a low-dimensional subspace, enabling efficient communication and computation. FLowDUP's learning objective is theoretically motivated by our new transductive multi-task PAC-Bayesian generalization bound, that provides performance guarantees for unlabeled clients. The objective is structured in such a way that it allows both clients with labeled data and clients with only unlabeled data to contribute to the training process. To supplement our theoretical results we carry out a thorough experimental evaluation of FLowDUP, demonstrating strong empirical performance on a range of datasets with differing sorts of statistically heterogeneous clients. Through numerous ablation studies, we test the efficacy of the individual components of the method.

Figures

Figures reproduced from arXiv: 2505.15579 by the authors.

**Figure 1.** Visualization of the client embeddings on Rotated Fashion-MNIST. Projection to two dimensions using PCA (left) and t-SNE (right). Understanding dataset embeddings Recall the hypernetwork architecture from (3). We can interpret the input to h2 as an embedding representation of the client dataset X, which we call r(X) := 1 |X| X x∈X h1(x). (9) To better understand the inner workings of this mechanism we visualize the… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 8 canonical work pages

[4]

Kairouz, P., McMahan, H

URL http://arxiv.org/abs/1909.12488. Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., D’Oliveira, R. G. L., Eichner, H., Rouayheb, S. E., Evans, D., Gardner, J., Garrett, Z., Gascón, A., Ghazi, B., Gibbons, P. B., Gruteser, M., Harchaoui, Z., He, C., He, L., Huo, Z., Hutc...

arXiv 1909
[9]

Zakerinia, H., Ghobadi, D., and Lampert, C. H. From low intrinsic dimensionality to non-vacuous generalization bounds in deep multi-task learning. arXiv preprint arXiv:2501.19067,

arXiv
[11]

we have a distribution D, and the goal is to generalize from training data to the unseen data

we have that for any prior P , and any δ >0, over the sampling of the dataset S for all posteriors Q it holds: R(Q)) ≤ bR(Q) + s KL(Q∥P ) + log(2√mL δ ) 2mL , (10) This bound holds in an inductive learning setting, i.e. we have a distribution D, and the goal is to generalize from training data to the unseen data. On the other hand, in transductive learnin...

work page 2014
[14]

Parameter decomposition-approaches (Arivazhagan et al., 2019; Collins et al., 2021; Marfoq et al., 2022; Chen et al., 2023; Wu et al.,

which learn a single global model that can be easily personalized using a small number of gradient steps on the client. Parameter decomposition-approaches (Arivazhagan et al., 2019; Collins et al., 2021; Marfoq et al., 2022; Chen et al., 2023; Wu et al.,

work page 2019
[15]

Federated multi-task approaches (Smith et al., 2017; Dinh et al., 2020; Hanzely et al., 2020; Marfoq et al., 2021; Li et al., 2021; Lin et al., 2022; Ye et al., 2023; Zhang et al.,

which divide the learnable parameters into some that are shared across clients (such as a feature extractor) and some that are specific individual to each client (such as a classification head). Federated multi-task approaches (Smith et al., 2017; Dinh et al., 2020; Hanzely et al., 2020; Marfoq et al., 2021; Li et al., 2021; Lin et al., 2022; Ye et al., 2...

work page 2017
[16]

All of the above approaches, however, require a client to have labeled data in order to obtain a personalized model

learn separate models for each client while still sharing some information across clients, for instance by regularizing towards some global model. All of the above approaches, however, require a client to have labeled data in order to obtain a personalized model. Learning in a subspace This formulation of intrinsic dimensionality and learning in a subspac...

work page 2018
[256]

For h2 we always use a fully connected network with a single hidden layer and ReLU non-linearity. C Additional Related Work Personalized federated learning Approaches to personalized FL typically fall into one of the following categories: Meta-learning based approaches (Jiang et al., 2019; Fallah et al.,

work page 2019
[500]

For all methods we tune the local learning rate ηl on validation clients

All methods use a client cohort size each round of size 100, a local number of epochs set to E = 1 and a local batch size of B = 20 for FEMNIST and B = 50 on all other datasets. For all methods we tune the local learning rate ηl on validation clients. Regarding method specific hyperparameters. For FedProx, we set µ following Li et al. (2020), that is µ = ...

work page 2020

Show all 15 references

[2004]

Maurer, A

URL http: //arxiv.org/abs/cs.LG/0411099. Maurer, A. Bounds for linear multi-task learning. Journal of Machine Learning Research (JMLR), 7: 117–139,
[2017]

Ye, R., Ni, Z., Wu, F., Chen, S., and Wang, Y

URL http://arxiv.org/abs/1708.07747. Ye, R., Ni, Z., Wu, F., Chen, S., and Wang, Y . Personalized federated learning with inferred collaboration graphs. In International Conference on Machine Learning (ICML),

arXiv
[2018]

Caruana, R

URL http://arxiv.org/abs/1812.01097. Caruana, R. Multitask learning. Machine Learning, 28:41–75,

arXiv
[2019]

Bartlett, P

URL http://arxiv.org/abs/1912.00818. Bartlett, P. L. and Mendelson, S. Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research (JMLR),

1912 arXiv
[2020]

Zhang, H., Li, C., Dai, W., Zou, J., and Xiong, H

URL https://arxiv.org/pdf/2010.08982. Zhang, H., Li, C., Dai, W., Zou, J., and Xiong, H. FedCR: Personalized federated learning based on across-client common representation with conditional mutual information regularization. In International Conference on Machine Learning (ICML),

2010 arXiv
[2023]

UPFL: unsupervised personalized federated learning towards new clients

Ye, T., Chen, C., Wang, Y ., Li, X., and Gao, M. UPFL: unsupervised personalized federated learning towards new clients. In Proceedings of the 2024 SIAM International Conference on Data Mining, SDM,

2024
[2024]

URL https://arxiv.org/abs/2404.06430. Guan, J. and Lu, Z. Fast-rate PAC-Bayesian generalization bounds for meta-learning. InInternational Conference on Machine Learning (ICML),

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [4]

Kairouz, P., McMahan, H

URL http://arxiv.org/abs/1909.12488. Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., D’Oliveira, R. G. L., Eichner, H., Rouayheb, S. E., Evans, D., Gardner, J., Garrett, Z., Gascón, A., Ghazi, B., Gibbons, P. B., Gruteser, M., Harchaoui, Z., He, C., He, L., Huo, Z., Hutc...

arXiv 1909

[2] [9]

Zakerinia, H., Ghobadi, D., and Lampert, C. H. From low intrinsic dimensionality to non-vacuous generalization bounds in deep multi-task learning. arXiv preprint arXiv:2501.19067,

arXiv

[3] [11]

we have a distribution D, and the goal is to generalize from training data to the unseen data

we have that for any prior P , and any δ >0, over the sampling of the dataset S for all posteriors Q it holds: R(Q)) ≤ bR(Q) + s KL(Q∥P ) + log(2√mL δ ) 2mL , (10) This bound holds in an inductive learning setting, i.e. we have a distribution D, and the goal is to generalize from training data to the unseen data. On the other hand, in transductive learnin...

work page 2014

[4] [14]

Parameter decomposition-approaches (Arivazhagan et al., 2019; Collins et al., 2021; Marfoq et al., 2022; Chen et al., 2023; Wu et al.,

which learn a single global model that can be easily personalized using a small number of gradient steps on the client. Parameter decomposition-approaches (Arivazhagan et al., 2019; Collins et al., 2021; Marfoq et al., 2022; Chen et al., 2023; Wu et al.,

work page 2019

[5] [15]

Federated multi-task approaches (Smith et al., 2017; Dinh et al., 2020; Hanzely et al., 2020; Marfoq et al., 2021; Li et al., 2021; Lin et al., 2022; Ye et al., 2023; Zhang et al.,

which divide the learnable parameters into some that are shared across clients (such as a feature extractor) and some that are specific individual to each client (such as a classification head). Federated multi-task approaches (Smith et al., 2017; Dinh et al., 2020; Hanzely et al., 2020; Marfoq et al., 2021; Li et al., 2021; Lin et al., 2022; Ye et al., 2...

work page 2017

[6] [16]

All of the above approaches, however, require a client to have labeled data in order to obtain a personalized model

learn separate models for each client while still sharing some information across clients, for instance by regularizing towards some global model. All of the above approaches, however, require a client to have labeled data in order to obtain a personalized model. Learning in a subspace This formulation of intrinsic dimensionality and learning in a subspac...

work page 2018

[7] [256]

For h2 we always use a fully connected network with a single hidden layer and ReLU non-linearity. C Additional Related Work Personalized federated learning Approaches to personalized FL typically fall into one of the following categories: Meta-learning based approaches (Jiang et al., 2019; Fallah et al.,

work page 2019

[8] [500]

For all methods we tune the local learning rate ηl on validation clients

All methods use a client cohort size each round of size 100, a local number of epochs set to E = 1 and a local batch size of B = 20 for FEMNIST and B = 50 on all other datasets. For all methods we tune the local learning rate ηl on validation clients. Regarding method specific hyperparameters. For FedProx, we set µ following Li et al. (2020), that is µ = ...

work page 2020