Pith. sign in

REVIEW 5 major objections 4 minor 39 references

Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning

T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Federated RL clients can generalize across environments by sharing only a behavior-metric state projection, keeping raw observations private.

desk verdict Plausible empirical extension—sharing RAG-based state projection parameters in federated RL—but the privacy claim overreaches and the FedAvg baseline is not what it claims to be. read the letter →

arxiv 2505.09959 v1 pith:LS46N7JB submitted 2025-05-15 cs.LG

Zengxia Guo , Bohui An , Zhongqi Lu This is my paper · ORCID
classification cs.LG
keywords federatedreinforcementlearningbehavioralmetricstateprojectionrepresentationheterogeneousenvironmentsprivacy-preservingRAGdistanceSoftActor-Critic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedRAG aims to show that federated reinforcement learning does not need to share raw states, rewards, or policies to help clients learn from one another. Each client learns a state-projection function that maps high-dimensional observations into a space where distances approximate a behavioral metric, and only the parameters of these projection functions are averaged on a central server. The paper argues that this sharing improves cross-environment generalization on DeepMind Control tasks and, as a side effect, protects sensitive task-specific information. If true, the proposal separates what is shared, a task-relevant representation, from what remains private, raw observations and transition data, which is the core trade-off federated RL must manage.

What carries the argument

The load-bearing object is the approximated behavioral-metric state projection function $\phi_\omega$, trained with the RAG distance. RAG is a recursive behavioral metric: the distance between two states is the difference in expected reward under the policy plus a discounted distance between expected next states; the paper approximates the intractable reward-variance term with a learned Gaussian reward model and the next-state term with a learned Gaussian dynamics model, and uses a MICo-style embedding distance with nonzero self-distance. This projection compresses each client's high-dimensional observation into task-relevant features, and it is the only object whose parameters are shared, averaged, and regularized in federated learning.

What would settle it

Run a model-inversion or gradient-leakage attack on the uploaded $\omega_k$ in the natural-video CartPole setup from Appendix A.3: if frames from a client's local replay buffer can be reconstructed or classified above chance from the shared projection parameters, the claim that $\omega$ has 'nothing to do with our private data state' is refuted. A positive estimate of mutual information between $\omega_k$ and local observations would also settle the question.

Watch

Extended reading notes

Core claim

The central claim is that aggregating the parameters $\omega$ of locally learned behavioral-metric state projections gives each federated RL client useful knowledge from the others without exchanging any raw state, reward, or transition data. Each client's projection $\phi_{\omega_k}$ is trained so that embedding distances approximate the Reducing Approximation Gap (RAG) distance, a behavioral metric comparing expected rewards and expected next states under the local policy. The server forms a global projection by averaging the local $\omega_k$'s, and each client's loss adds an L2 pull toward that global value while periodically replacing local parameters with global ones. The paper reports better performance in other, unseen, and visually distracted environments than local training, FedAvg, or FeSAC, and concludes that sharing projection functions gives each client information gain while task-specific data never leaves the client.

Load-bearing premise

The privacy guarantee rests on the assertion that the shared projection parameters $\omega$ are unrelated to the raw private states; if an adversary can recover state features from those parameters, the paper's central privacy claim collapses.

Editorial extensions

If this is right

  • A client trained in one physical configuration can act in previously unseen variants of its task, because the shared projection encodes task-relevant behavior rather than environment-specific pixels.
  • Increasing environmental heterogeneity among clients no longer degrades cross-environment performance as sharply as in FedAvg, since the behavioral metric filters environment-specific noise.
  • Communication cost is reduced: only lightweight projection parameters are exchanged, not raw states, transitions, or full policy gradients.
  • Tuning the regularization weight $\lambda$ controls a concrete trade-off: larger values improve out-of-environment generalization, while very large values slow local adaptation and hurt same-environment performance.
  • Replacing local parameters with global ones and applying L2 regularization are both necessary; the paper's ablation shows removing either component causes significant performance drops.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: run model-inversion or gradient-based attacks on the aggregated projection parameters; if local observation features can be recovered, the privacy side-effect would need explicit defenses such as noise injection or encryption.
  • The same parameter-sharing recipe could be applied to other behavioral metrics, such as bisimulation or contrastive similarities, since the mechanism only requires a trainable state distance; the paper demonstrates one metric.
  • The paper assumes the global projection parameters follow a Gaussian and averages them; comparing this with robust aggregators would show whether the gains come from averaging itself or from the particular shape assumed for the shared model distribution.
  • Because the projection is trained to discard task-irrelevant background, the approach suggests a broader link between behavioral regularization and privacy: representations that ignore environmental noise may leak less, a connection the paper does not quantify.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes FedRAG, a federated reinforcement learning framework in which each client learns a state projection function fitted to an approximate behavioral metric (the RAG distance) and only the parameters of this projection function are shared with and aggregated by a central server. Clients train local SAC policies on the projected states, with an L2 penalty pulling local projection parameters toward the global average. The authors claim that sharing the projection-function parameters improves cross-environment generalization and simultaneously protects privacy by avoiding the exchange of raw states, rewards, and dynamics. Experiments on modified DeepMind Control Suite tasks (cartpole, cheetah, finger, walker) compare FedRAG with a baseline labeled FedAvg, with FeSAC, and with local-only training, reporting improved performance in other-environment evaluation. The appendix includes proofs that the RAG distance is a contraction and that the distance upper-bounds the value-function difference.

Significance. Representation sharing through a behavioral-metric encoder is a plausible and relatively underexplored idea in federated RL, and the paper's empirical results, if reproducible, would demonstrate that such sharing can improve cross-environment generalization with low communication cost. The simplicity of the method (parameter averaging plus L2 regularization) is a practical strength. However, the paper's strongest advertised claim—that FedRAG 'shares no sensitive task-specific information'—is not established: the encoder is trained on private states and the shared parameters can encode information about the local state distribution, yet no formal privacy guarantee or attack evaluation is provided. The performance claim is also weakened by a nonstandard FedAvg baseline and by selection of the key hyperparameter lambda on the evaluation metric. With these issues addressed, the method could be a useful contribution to federated representation learning for RL; in its current form, the significance is largely conditional.

major comments (5)
  1. [Section 4.4] The privacy claim is not supported and is internally inconsistent. The text asserts that omega 'is only related to the mapped state and reward, and has nothing to do with our private data state,' but Eq. (15) defines L_FedRAG as an expectation over D_k, which contains raw private states (s, a, r, s'), and the representation phi_omega(s) is a deterministic function of the private state s. The gradient of Eq. (15) with respect to omega therefore depends on the raw state distribution, so sharing omega can leak information about local state observations (e.g., via model inversion or membership inference). The paper provides no formal privacy guarantee (e.g., differential privacy), no noise mechanism, and no attack evaluation. The abstract's assertion that FedRAG 'shares no sensitive task-specific information' should be withdrawn or replaced by a bounded statement such as 'does not directly transmit raw states or rewards'; otherwise the central claim is unsupported.
  2. [Section 5.2] The FedAvg baseline is defined as FedRAG with lambda=0, but this is not standard FedAvg. With lambda=0, the client still optimizes the RAG loss in Eq. (13) and the global encoder parameters are still averaged; this is a representation-learning variant, not the standard federated averaging of policy or Q networks. Consequently, the reported improvement over 'FedAvg' may be attributable to the behavioral-metric representation loss rather than to the proposed collaboration mechanism. A proper FedAvg baseline (e.g., federated averaging of the SAC actor-critic parameters, or of the encoder without the RAG loss) is needed to support the claim that FedRAG's specific design is responsible for the gains.
  3. [Section 5.3 and Figs. 3-6] The regularization weight lambda is selected on the same evaluation metric used to report the final results. Section 5.3 reports that 'the optimal performance was achieved at lambda=0.001' based on performance in other environments, and this value is then reused in Figs. 4, 5, 6, 9, and 10. Since the selection is made on the test environments, the reported cross-environment improvements may reflect hyperparameter fitting rather than a general property of the algorithm. The paper should either report a separate validation split or demonstrate that the gains are stable across a range of lambda values.
  4. [Equation (13)] Equation (13) as displayed defines L_RAG as a difference of two squared terms, i.e., (d_hat(...) - gamma*d_hat(...))^2 - (|r_i - r_j|^2 - sigma_i^2 - sigma_j^2)^2. This objective is not a valid regression loss: it can be negative, is unbounded below, and does not become zero when the estimated distance matches the RAG distance. The intended loss from the RAG literature should be the square of the difference between the embedding distance and the target metric (including the square-rooted reward-variance term), not the difference of two independent squared terms. If this is a typesetting error, the correct expression must be given, because Algorithm 1 and Eq. (15) inherit the same malformed form.
  5. [Sections 5.1-5.6 and Appendix A] The manuscript does not state the number of random seeds, does not show error bars or confidence intervals in most figures, and does not provide code or a description of the compute environment. The text in Section 5.2 mentions 'standard deviation,' but the figures appear to show single curves without uncertainty bands. Without these details, the claimed improvements cannot be distinguished from training variance, which is essential for an empirical paper in this area.
minor comments (4)
  1. [Section 5.1 and Table 1] The main text states that an episode consists of 125 environment steps and that 4000 episodes yield 500,000 steps, but Table 1 lists 'Episode length 1000.' Please reconcile this inconsistency.
  2. [Table 1 and Eq. (14)] The hyperparameter K in the MICo-style distance (Eq. 14) is introduced as a hyperparameter but is not listed in Table 1, despite the table reporting other hyperparameters such as alpha_RAG and alpha_P. Please report its value or explain how it is set.
  3. [Section 4.3] The justification 'According to the central limit theorem, we approximate the global Gaussian distribution by aggregating the mean of all local omega_k' is imprecise: averaging parameters does not follow from the central limit theorem, and the notion of a 'global Gaussian distribution' over parameters is not defined. This should be reworded as a design choice.
  4. [Section 5.1] The synchronization period is described as 'every 4 episodes,' which seems short relative to one episode of 1000 steps; if this is a typo, the intended number of local update steps should be specified.

Circularity Check

1 steps flagged · score 6.0 of 10

Privacy claim in §4.4 is self-definitional: shared ω is said to depend only on 'mapped states' that are themselves functions of private raw states, so the no-leakage conclusion is assumed rather than derived.

  1. self definitional [Section 4.4 (Effectiveness of Anti-attack), after Eq. 15 and Eq. 16]
    "From the loss L_FedRAG(ϕω) in Equation 15, we can also see that ω is only related to the mapped state and reward, and has nothing to do with our private data state. Therefore, our proposed FedRAG protects the privacy of local state information to a certain extent."

    The 'mapped state' is defined as φω(s) with s drawn from the private local buffer D_k, and Eq. 15 trains φω on exactly those private states and rewards. The premise that ω is unrelated to private data because it depends only on mapped states and rewards is therefore not independent of the private data: the mapped state is a function of it. The privacy conclusion holds only if one defines φω(s) as non-sensitive by construction, which is precisely the claim being proved. No leakage bound, differential privacy mechanism, or attack evaluation is supplied to break the dependency, so the argument reduces to renaming the private input as 'mapped state' and then asserting that the renamed object is not private.

full rationale

FedRAG's performance contribution is not circular: the RAG encoder and the federated averaging mechanism are taken from prior work (Chen and Pan 2022; Castro et al. 2021) and evaluated on external DeepMind Control benchmarks, and no performance prediction is derived by fitting constants to the reported curves. The hyperparameter λ is selected on the same evaluation metric in Section 5.3, which is a tuning caveat but not a circular reduction, because the empirical comparisons are not forced by λ's definition. The one genuine circular step is the privacy proof in Section 4.4: the argument that shared ω is private because it 'only' depends on mapped states and rewards assumes that mapped states carry no private information, even though mapped states are defined as φω(s) of the private raw state. Without a quantitative leakage bound or attack evaluation, the privacy conclusion is equivalent to the premise that projection removes sensitivity. Thus the score reflects partial circularity: the privacy claim reduces by definition, while the RL performance results remain independent and externally evaluated.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. The only invention is the FedRAG algorithm itself, which uses existing components, so the axiom ledger captures the borrowed and assumed pieces that the central claim depends on.

free parameters (4)
  • lambda (L2 regularization weight) = 0.001
    Selected in Section 5.3 as the value with best performance in other environments; a data-fitted hyperparameter.
  • K (MICo distance scaling) = not specified
    Introduced in Eq. 14 as a hyperparameter; value not reported, limiting reproducibility.
  • alpha_RAG = 0.5
    Listed in Table 1 but never appears in the displayed loss equations; presumably a loss weight in implementation.
  • alpha_P = 0.0001
    Listed in Table 1 but never appears in the displayed loss equations; presumably a loss weight for the dynamics model.
assumptions (5)
  • standard math Banach fixed point theorem
    Used in Appendix B to prove Theorem 1 that d_pi has a unique fixed point.
  • domain assumption Central limit theorem justifies averaging local parameters to approximate a global Gaussian distribution
    Section 4.3: 'According to the central limit theorem, we approximate the global Gaussian distribution by aggregating the mean of all local omega_k'. This is not justified because CLT applies to sums of random variables, not parameter vectors.
  • domain assumption Reward r(s) is Gaussian distributed
    Section 4.2: 'assuming that the reward r^s on state s is Gaussian distributed' to justify learning a mean/variance approximator.
  • domain assumption Clients' MDPs share action space and discount factor, and state projections map into a common embedding
    Section 4.1 problem formulation; needed for parameter averaging to be meaningful across clients.
  • domain assumption RAG distance and its approximation from Chen and Pan (2022) are valid
    The paper imports Eq. 9-10 from prior work without re-derivation; the central representation learning relies on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning." pith.science (2026). https://pith.science/paper/LS46N7JB

@misc{pith2026250509959,
  author       = {Pith},
  title        = {Pith review of: Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LS46N7JB}},
  note         = {Machine review of arXiv:2505.09959}
}
read the original abstract

Federated reinforcement learning (FRL) methods usually share the encrypted local state or policy information and help each client to learn from others while preserving everyone's privacy. In this work, we propose that sharing the approximated behavior metric-based state projection function is a promising way to enhance the performance of FRL and concurrently provides an effective protection of sensitive information. We introduce FedRAG, a FRL framework to learn a computationally practical projection function of states for each client and aggregating the parameters of projection functions at a central server. The FedRAG approach shares no sensitive task-specific information, yet provides information gain for each client. We conduct extensive experiments on the DeepMind Control Suite to demonstrate insightful results.

Figures

Figures reproduced from arXiv: 2505.09959 by the authors.

Figure 1
Figure 1. Framework of FedRAG. Periodically, the local state pro [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. The results of varying lambda. In the above experiment, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 2
Figure 2. Comparison of FedRAG with Baseline in other environ [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of Local and FedRAG with λ = 0.1/0.001 in same or other environments. environments. The Local approach limits clients to their own environments, resulting in local optimal policies that poorly generalize. In contrast, FedRAG aggregates local state pro￾jectio…
Figure 6
Figure 6. Figure 6: Comparison of FedRAG and FedAvg across different [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: In cartpole-swing and finger-spin tasks, FedRAG [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: Illustrations of observations in DMC cartpole-swingup [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Performance comparison of FedRAG and FedAvg with [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 7
Figure 7. Figure 7: Performance comparison of FedRAG and variants in other [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 10
Figure 10. Figure 10: Comparison of FedRAG and Baseline in unseen environ [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Illustrations of observations in DMC tasks for pole [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 26 canonical work pages

  1. [1]

    Deep learning with differential privacy

    [Abadiet al., 2016 ] Martin Abadi, Andy Chu, Ian Goodfel- low, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on com- puter and communications security, pages 308–318,

    work page 2016

  2. [4]

    Mico: Im- proved representations via sampling-based state similarity for markov decision processes.Advances in Neural Infor- mation Processing Systems, 34:30113–30126,

    [Castroet al., 2021 ] Pablo Samuel Castro, Tyler Kastner, Prakash Panangaden, and Mark Rowland. Mico: Im- proved representations via sampling-based state similarity for markov decision processes.Advances in Neural Infor- mation Processing Systems, 34:30113–30126,

    work page 2021

  3. [5]

    Scalable methods for computing state similarity in deterministic markov deci- sion processes

    [Castro, 2020] Pablo Samuel Castro. Scalable methods for computing state similarity in deterministic markov deci- sion processes. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10069–10076,

    work page 2020

  4. [8]

    Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach.Advances in neural information processing sys- tems, 33:3557–3568,

    [Fallahet al., 2020 ] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach.Advances in neural information processing sys- tems, 33:3557–3568,

    work page 2020

  5. [9]

    Fault-tolerant federated reinforcement learning with the- oretical guarantee.Advances in Neural Information Pro- cessing Systems, 34:1007–1021,

    [Fanet al., 2021 ] Xiaofeng Fan, Yining Ma, Zhongxiang Dai, Wei Jing, Cheston Tan, and Bryan Kian Hsiang Low. Fault-tolerant federated reinforcement learning with the- oretical guarantee.Advances in Neural Information Pro- cessing Systems, 34:1007–1021,

    work page 2021

  6. [10]

    Fedhql: Federated heterogeneous q-learning

    [Fanet al., 2023 ] Flint Xiaofeng Fan, Yining Ma, Zhongxi- ang Dai, Cheston Tan, Bryan Kian Hsiang Low, and Roger Wattenhofer. Fedhql: Federated heterogeneous q-learning. arXiv preprint arXiv:2301.11135,

    arXiv 2023

  7. [12]

    Bisimulation metrics for continuous markov decision processes.SIAM Journal on Computing, 40(6):1662–1714,

    [Fernset al., 2011 ] Norm Ferns, Prakash Panangaden, and Doina Precup. Bisimulation metrics for continuous markov decision processes.SIAM Journal on Computing, 40(6):1662–1714,

    work page 2011

  8. [15]

    Federated reinforcement learn- ing with environment heterogeneity

    [Jinet al., 2022 ] Hao Jin, Yang Peng, Wenhao Yang, Shusen Wang, and Zhihua Zhang. Federated reinforcement learn- ing with environment heterogeneity. InInternational Con- ference on Artificial Intelligence and Statistics, pages 18–

    work page 2022

Show all 39 references
  1. [16]

    Towards robust bisimulation metric learning.Advances in Neural Infor- mation Processing Systems, 34:4764–4777,

    [Kemertas and Aumentado-Armstrong, 2021] Mete Kemer- tas and Tristan Aumentado-Armstrong. Towards robust bisimulation metric learning.Advances in Neural Infor- mation Processing Systems, 34:4764–4777,

    2021

  2. [17]

    Think locally, act globally: Federated learning with local and global representations.arXiv preprint arXiv:2001.01523,

    [Lianget al., 2020 ] Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Rus- lan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations.arXiv preprint arXiv:2001.01523,

    2020 arXiv

  3. [18]

    Policy-independent behavioral metric-based rep- resentation for deep reinforcement learning.Proceed- ings of the AAAI Conference on Artificial Intelligence, 37:8746–8754, 06

    [Liaoet al., 2023 ] Weijian Liao, Zongzhang Zhang, and Yang Yu. Policy-independent behavioral metric-based rep- resentation for deep reinforcement learning.Proceed- ings of the AAAI Conference on Artificial Intelligence, 37:8746–8754, 06

    2023

  4. [19]

    Threats to federated learning: A survey,

    [Lyuet al., 2020 ] Lingjuan Lyu, Han Yu, and Qiang Yang. Threats to federated learning: A survey,

    2020

  5. [20]

    Communication-efficient learning of deep networks from decentralized data

    [McMahanet al., 2017 ] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–1282. PMLR,

    2017

  6. [22]

    A survey on security and privacy of federated learning.Future Generation Computer Systems, 115:619–640,

    [Mothukuriet al., 2021 ] Viraaji Mothukuri, Reza M Parizi, Seyedamin Pouriyeh, Yan Huang, Ali Dehghantanha, and Gautam Srivastava. A survey on security and privacy of federated learning.Future Generation Computer Systems, 115:619–640,

    2021

  7. [23]

    Privacy-preserving federated learning using homomorphic encryption.Applied Sciences, 12(2):734,

    [Park and Lim, 2022] Jaehyoung Park and Hyuk Lim. Privacy-preserving federated learning using homomorphic encryption.Applied Sciences, 12(2):734,

    2022

  8. [24]

    Federated reinforcement learning: Techniques, applications, and open challenges.arXiv preprint arXiv:2108.11887,

    [Qiet al., 2021 ] Jiaju Qi, Qihao Zhou, Lei Lei, and Kan Zheng. Federated reinforcement learning: Techniques, applications, and open challenges.arXiv preprint arXiv:2108.11887,

    2021 arXiv

  9. [25]

    Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

    [Reddiet al., 2020 ] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Koneˇcn`y, Sanjiv Kumar, and H Brendan McMa- han. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

    2020 arXiv

  10. [26]

    Personalized federated learning with moreau en- velopes.Advances in neural information processing sys- tems, 33:21394–21405,

    [T Dinhet al., 2020 ] Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau en- velopes.Advances in neural information processing sys- tems, 33:21394–21405,

    2020

  11. [27]

    Federated learning from pre-trained models: A contrastive learning ap- proach.Advances in neural information processing sys- tems, 35:19332–19344,

    [Tanet al., 2022 ] Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning ap- proach.Advances in neural information processing sys- tems, 35:19332–19344,

    2022

  12. [28]

    Fesac: Federated learning-based soft actor-critic traffic offloading in space-air-ground integrated network.arXiv preprint arXiv:2212.02075,

    [Tanget al., 2022 ] Fengxiao Tang, Yilin Yang, Xin Yao, Ming Zhao, and Nei Kato. Fesac: Federated learning-based soft actor-critic traffic offloading in space-air-ground integrated network.arXiv preprint arXiv:2212.02075,

    2022 arXiv

  13. [29]

    Deepmind control suite.arXiv preprint arXiv:1801.00690,

    [Tassaet al., 2018 ] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite.arXiv preprint arXiv:1801.00690,

    2018 arXiv

  14. [30]

    A hybrid approach to privacy-preserving feder- ated learning

    [Truexet al., 2019 ] Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, and Yi Zhou. A hybrid approach to privacy-preserving feder- ated learning. InProceedings of the 12th ACM workshop on artificial intelligence and security, pages 1–11,

    2019

  15. [32]

    Turbosvm-fl: Boost- ing federated learning through svm aggregation for lazy clients

    [Wanget al., 2024 ] Mengdi Wang, Anna Bodonhelyi, Efe Bozkir, and Enkelejda Kasneci. Turbosvm-fl: Boost- ing federated learning through svm aggregation for lazy clients. InProceedings of the AAAI Conference on Artifi- cial Intelligence, volume 38, pages 15546–15554,

    2024

  16. [34]

    Learn- ing invariant representations for reinforcement learning without reconstruction.CoRR, abs/2006.10742,

    [Zhanget al., 2020a ] Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learn- ing invariant representations for reinforcement learning without reconstruction.CoRR, abs/2006.10742,

    2006 arXiv

  17. [35]

    No free lunch theorem for se- curity and utility in federated learning.ACM Transactions on Intelligent Systems and Technology, 14(1):1–35,

    [Zhanget al., 2022 ] Xiaojin Zhang, Hanlin Gu, Lixin Fan, Kai Chen, and Qiang Yang. No free lunch theorem for se- curity and utility in federated learning.ACM Transactions on Intelligent Systems and Technology, 14(1):1–35,

    2022

  18. [36]

    Federated unsupervised representation learning.Frontiers of Information Technol- ogy & Electronic Engineering, 24(8):1181–1193,

    [Zhanget al., 2023 ] Fengda Zhang, Kun Kuang, Long Chen, Zhaoyang You, Tao Shen, Jun Xiao, Yin Zhang, Chao Wu, Fei Wu, Yueting Zhuang, et al. Federated unsupervised representation learning.Frontiers of Information Technol- ogy & Electronic Engineering, 24(8):1181–1193,

    2023

  19. [37]

    Fed- erated learning with non-iid data.arXiv preprint arXiv:1806.00582,

    [Zhaoet al., 2018 ] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Fed- erated learning with non-iid data.arXiv preprint arXiv:1806.00582,

    2018 arXiv

  20. [38]

    Deep leakage from gradients,

    [Zhuet al., 2019 ] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients,

    2019

  21. [39]

    Federated deep reinforcement learning.arXiv preprint arXiv:1901.08277,

    [Zhuoet al., 2019 ] Hankz Hankui Zhuo, Wenfeng Feng, Yufeng Lin, Qian Xu, and Qiang Yang. Federated deep reinforcement learning.arXiv preprint arXiv:1901.08277,

    2019 arXiv

  22. [2011]

    Differentially private federated learning: A client level perspective.arXiv preprint arXiv:1712.07557,

    [Geyeret al., 2017 ] Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective.arXiv preprint arXiv:1712.07557,

    2017 arXiv

  23. [2016]

    Contrastive behavioral similarity embeddings for generalization in reinforcement learning.arXiv preprint arXiv:2101.05265,

    [Agarwalet al., 2021 ] Rishabh Agarwal, Marlos C Machado, Pablo Samuel Castro, and Marc G Belle- mare. Contrastive behavioral similarity embeddings for generalization in reinforcement learning.arXiv preprint arXiv:2101.05265,

    2021 arXiv

  24. [2017]

    Aby3: A mixed protocol framework for machine learning

    [Mohassel and Rindal, 2018] Payman Mohassel and Peter Rindal. Aby3: A mixed protocol framework for machine learning. InProceedings of the 2018 ACM SIGSAC con- ference on computer and communications security, pages 35–52,

    2018

  25. [2018]

    Soft actor-critic algorithms and applications

    [Haarnojaet al., 2018b ] Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905,

    arXiv

  26. [2019]

    Optimizing federated learning on non- iid data with reinforcement learning

    [Wanget al., 2020 ] Hao Wang, Zakhary Kaplan, Di Niu, and Baochun Li. Optimizing federated learning on non- iid data with reinforcement learning. InIEEE INFO- COM 2020-IEEE conference on computer communica- tions, pages 1698–1707. IEEE,

    2020

  27. [2020]

    Learn- ing representations via a robust behavioral metric for deep reinforcement learning.Advances in Neural Information Processing Systems, 35:36654–36666,

    [Chen and Pan, 2022] Jianda Chen and Sinno Pan. Learn- ing representations via a robust behavioral metric for deep reinforcement learning.Advances in Neural Information Processing Systems, 35:36654–36666,

    2022

  28. [2021]

    Multi-task federated reinforcement learn- ing with adversaries.CoRR, abs/2103.06473,

    [Anwar and Raychowdhury, 2021] Aqeel Anwar and Arijit Raychowdhury. Multi-task federated reinforcement learn- ing with adversaries.CoRR, abs/2103.06473,

    2021 arXiv

  29. [2022]

    Exploiting shared repre- sentations for personalized federated learning

    [Collinset al., 2021 ] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared repre- sentations for personalized federated learning. InInterna- tional conference on machine learning, pages 2089–2099. PMLR,

    2021

  30. [2023]

    Pri- vacy preserving machine learning with homomorphic en- cryption and federated learning.Future Internet, 13(4):94,

    [Fang and Qian, 2021] Haokun Fang and Quan Qian. Pri- vacy preserving machine learning with homomorphic en- cryption and federated learning.Future Internet, 13(4):94,

    2021

  31. [2024]

    Federated learning with dif- ferential privacy: Algorithms and performance analysis

    [Weiet al., 2020 ] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor. Federated learning with dif- ferential privacy: Algorithms and performance analysis. IEEE transactions on information forensics and security, 15:...

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.