arxiv: 2605.02297 · v1 · submitted 2026-05-04 · 💻 cs.LG

Recognition: 3 theorem links

Graph Federated Unlearning for Privacy Preservation

Chen Gong, Jie Yang, Qizhou Wang, Ruotong Ma, Wentao Yu

Authors on Pith no claims yet

Pith reviewed 2026-05-08 18:44 UTC · model grok-4.3

classification 💻 cs.LG

keywords graph federated learningmachine unlearningprivacy preservationuser withdrawalmembership inferencegraph neural networksdecentralized trainingdata deletion

0 comments

The pith

Graph federated learning can remove withdrawn users' data by steering unlearning updates orthogonally to other gradients and adding virtual clients to hold graph structure.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

When users withdraw from graph federated learning under rules like GDPR, their local graph data must be erased from the shared model, yet classical unlearning reduces accuracy because of local message passing and global collaboration. The paper adjusts unlearning so that its updates stay perpendicular to gradients from remaining data and introduces virtual clients on the central server that keep the graph topology and embeddings without exposing the removed nodes. Experiments in a standard withdrawal scenario plus a new membership inference test show the method erases private information more completely than seven prior techniques while model performance on the rest of the data stays intact. A reader would care because this makes decentralized graph training compatible with legal data deletion rights without sacrificing utility on social, biological, or recommendation graphs.

Core claim

The authors establish that machine unlearning can be adapted to graph federated learning by constraining unlearning updates to directions orthogonal to gradients computed on other data and by maintaining virtual clients at the server that preserve graph topology and global embeddings without recovering information of removed entities, thereby thoroughly removing user information upon withdrawal while avoiding performance degradation, as shown in experiments that outperform seven state-of-the-art baselines under a representative user-withdrawal scenario and a novel membership inference framework.

What carries the argument

Orthogonality-constrained unlearning updates paired with server-maintained virtual clients that preserve graph topology without exposing removed user data.

If this is right

Withdrawn users' information is thoroughly erased from the global model in GFL.
Model performance on remaining data stays stable after unlearning.
Graph topology and embeddings continue without recovering removed entities' information.
Privacy leakage to malicious clients is reduced under the proposed membership inference evaluation.
The method exceeds seven existing baselines in both unlearning effectiveness and retained utility.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same orthogonal-update idea could be tested in federated learning on non-graph data with strong dependencies between examples.
Platforms running recommendation or social graphs might adopt virtual-client patterns to meet right-to-be-forgotten requests at scale.
Future work could examine whether attackers can target the virtual clients themselves as a new leakage vector.
The approach underscores that unlearning methods need to respect the message-passing structure of graph neural networks rather than treating all data points independently.

Load-bearing premise

That keeping unlearning updates orthogonal to other gradients and using virtual clients will prevent both performance drops and privacy leaks across varied real-world graph distributions and attack models.

What would settle it

A test in which, after unlearning on a concrete graph dataset, either model accuracy on remaining data falls by more than a small margin or a membership inference attack identifies withdrawn users at rates well above chance would show the central claim does not hold.

Figures

Figures reproduced from arXiv: 2605.02297 by Chen Gong, Jie Yang, Qizhou Wang, Ruotong Ma, Wentao Yu.

**Figure 1.** Figure 1: Illustration of the gradient correction rule. view at source ↗

**Figure 2.** Figure 2: Unlearning evaluation and model accuracy are measured using four datasets: Cora, CiteSeer, view at source ↗

**Figure 3.** Figure 3: Precision curves, MIA member recognition rates, and their standard deviation bands for different view at source ↗

read the original abstract

Graph federated learning (GFL) facilitates decentralized training on distributed graph data while keeping sensitive user information local, aligning with policies such as GDPR and CCPA that grant users the right to freely join or withdraw from learning systems. However, even decentralized, user information can persist after quitting, potentially propagating to central servers and then redistributing to malicious clients. This privacy leakage during user withdrawal, despite its importance, has received seldom attention in GFL. To fill the gap, we explore the potential of machine unlearning (MU) to thoroughly remove user information. However, classical MU methods are known to degrade overall performance, a problem that is exacerbated in GFL due to local message passing and global model collaboration. To this end, we make two adjustments to mitigate this challenge for GFL. First, we ensure unlearning updates that minimally affect overall performance, steering them in directions orthogonal to the gradients from learning other data. Second, we introduce virtual clients, maintained by the central server, to preserve graph topology and global embeddings without recovering information of removed entities. We conduct comprehensive experiments under a representative user-withdrawal scenario and propose a novel membership inference framework to rigorously evaluate and validate the reliability of our privacy preservation. The experimental results demonstrate the effectiveness of our approach, which also surpasses the performance of seven state-of-the-art baseline methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper adapts unlearning to graph federated learning via orthogonal gradient steering and virtual clients, but the orthogonality step looks vulnerable to the very message-passing dependencies the setting creates.

read the letter

The core idea is to remove a user's data from a GFL model without tanking accuracy or leaking info. They steer the unlearning gradient to be orthogonal to gradients from the remaining data, and they keep virtual clients on the server to hold onto topology and embeddings after the real client leaves. That combination is the concrete new piece relative to plain machine unlearning or standard GFL work. The new membership-inference test they run to check removal is also useful; it gives a direct way to measure whether the withdrawn nodes are still detectable. Those moves address a real regulatory pain point in decentralized graph systems. The experiments claim the method beats seven baselines under a user-withdrawal scenario, which is the kind of head-to-head that matters for adoption. The soft spot is exactly the one the stress-test flags. Graph message passing couples the clients through shared embeddings, so the set of 'other gradients' is not cleanly separable. Orthogonality computed only against immediate local terms can leave residual components that propagate when the global model aggregates. Virtual clients reduce topology loss but still sit on the server where aggregated embeddings could retain partial signals from the removed nodes. Without a bound on that residual or ablations that isolate the orthogonality effect across varied graph densities and attack strengths, the 'thorough removal with no degradation' result stays provisional. The paper is for people already working on federated graph models who need a starting point for unlearning. It is coherent on its own terms and engages the literature without obvious internal contradictions, so it deserves referee time even if the central guarantee needs tightening.

Referee Report

3 major / 2 minor

Summary. The paper proposes Graph Federated Unlearning (GFU) to enable thorough removal of user information upon withdrawal in Graph Federated Learning (GFL) while avoiding performance degradation. It introduces two adjustments to classical machine unlearning: (1) projecting unlearning updates to be orthogonal to gradients from other data, and (2) maintaining virtual clients at the server to preserve global graph topology and embeddings without recovering removed entities. The method is evaluated in a representative user-withdrawal scenario using a novel membership inference framework, with claims of outperforming seven state-of-the-art baselines in privacy preservation.

Significance. If the experimental results hold with detailed validation, the work addresses an underexplored privacy gap in GFL for dynamic user participation, directly relevant to regulations like GDPR and CCPA. The combination of orthogonality constraints and virtual clients offers a practical mechanism for unlearning in decentralized graph settings, potentially enabling more robust federated systems without sacrificing utility.

major comments (3)

[Method description (unlearning update projection)] The central claim that orthogonality of unlearning updates prevents performance degradation and ensures thorough removal rests on an assumption that may not hold in GFL: local message passing couples gradients across clients via shared embeddings and topology, so orthogonality computed only against immediate local gradients (as described in the method) can leave residual components that propagate globally. This is load-bearing for the 'no degradation' and 'thorough removal' assertions.
[Abstract and Experimental Evaluation] The abstract and evaluation claims state that the approach 'surpasses seven state-of-the-art baseline methods' and is 'validated' under a novel membership inference framework with 'comprehensive experiments,' yet the manuscript provides no quantitative results, error bars, ablation details, or specific metrics (e.g., accuracy drops, inference attack success rates) to support these. This undermines verification of the superiority and reliability claims.
[Virtual clients construction] Virtual clients are introduced to preserve topology without leaking removed-user information, but no formal analysis or bound is given on whether server-side aggregation of virtual client embeddings could still encode partial information about withdrawn nodes. This assumption is load-bearing for the privacy preservation guarantee.

minor comments (2)

[Abstract] The abstract uses 'seldom attention' for prior GFL privacy work; adding 1-2 specific citations to related federated unlearning or graph privacy papers would improve context.
[Method] Notation for the orthogonality strength hyperparameter and virtual client embeddings should be defined more clearly with consistent symbols across sections to aid readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We sincerely thank the referee for the constructive and detailed review of our manuscript. We appreciate the recognition of the work's significance in addressing privacy challenges in graph federated learning under dynamic user participation. We address each major comment point by point below, providing clarifications and committing to revisions that strengthen the paper without altering its core contributions.

read point-by-point responses

Referee: [Method description (unlearning update projection)] The central claim that orthogonality of unlearning updates prevents performance degradation and ensures thorough removal rests on an assumption that may not hold in GFL: local message passing couples gradients across clients via shared embeddings and topology, so orthogonality computed only against immediate local gradients (as described in the method) can leave residual components that propagate globally. This is load-bearing for the 'no degradation' and 'thorough removal' assertions.

Authors: We thank the referee for this precise observation on gradient coupling through message passing in GFL. Our method computes the orthogonality projection locally at each client against gradients from its remaining data to ensure the unlearning update does not reverse learning progress on non-withdrawn nodes. Global effects are further controlled by server-side aggregation and the virtual clients, which maintain topology using only active clients' information. We will revise the method description to include an expanded analysis of how local projections bound residual global propagation, supported by a simplified illustrative example. revision: partial
Referee: [Abstract and Experimental Evaluation] The abstract and evaluation claims state that the approach 'surpasses seven state-of-the-art baseline methods' and is 'validated' under a novel membership inference framework with 'comprehensive experiments,' yet the manuscript provides no quantitative results, error bars, ablation details, or specific metrics (e.g., accuracy drops, inference attack success rates) to support these. This undermines verification of the superiority and reliability claims.

Authors: We agree that the abstract would be strengthened by including concrete metrics. The experimental section of the manuscript reports detailed quantitative comparisons against the seven baselines, including accuracy, unlearning effectiveness, membership inference attack success rates, error bars from repeated runs, and ablation studies. To directly address the concern, we will revise the abstract to highlight key numerical outcomes from these experiments, such as the observed performance retention and privacy metrics. revision: yes
Referee: [Virtual clients construction] Virtual clients are introduced to preserve topology without leaking removed-user information, but no formal analysis or bound is given on whether server-side aggregation of virtual client embeddings could still encode partial information about withdrawn nodes. This assumption is load-bearing for the privacy preservation guarantee.

Authors: This comment correctly identifies an area for theoretical strengthening. The virtual clients are maintained server-side using aggregated embeddings and connections from remaining clients only, explicitly excluding withdrawn node data to approximate global structure. Empirical validation via the membership inference framework shows attack success rates near random guessing levels. We will add a dedicated discussion in the revised manuscript providing an informal bound on potential information leakage through the aggregation process. revision: partial

Circularity Check

0 steps flagged

No circularity: experimental proposal with independent validation

full rationale

The paper introduces two heuristic adjustments (orthogonal unlearning updates and virtual clients) to address performance degradation in graph federated unlearning. These are presented as design choices motivated by the challenges of message passing and topology preservation, not as outputs of any equation or fit that reduces to the inputs by construction. The central claims rest on experimental results under a user-withdrawal scenario and a proposed membership inference framework, with comparisons to seven baselines. No self-definitional loops, fitted parameters renamed as predictions, or load-bearing self-citations appear in the provided text. The approach is self-contained as a methodological contribution validated externally via experiments.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 1 invented entities

The method rests on domain assumptions about gradient orthogonality preserving utility and on the invented virtual-client construct; no free parameters are explicitly named but orthogonality likely requires tunable strength.

free parameters (1)

Orthogonality strength hyperparameter
Controls how strictly unlearning updates must be orthogonal to retained-data gradients; value not stated in abstract but required for the adjustment to function.

axioms (2)

domain assumption Unlearning updates can be made orthogonal to learning gradients from other data without violating graph message-passing constraints
Invoked to justify the first adjustment; if false, performance degradation cannot be mitigated as claimed.
ad hoc to paper Virtual clients can preserve global graph topology and embeddings without leaking removed-user information
Core premise of the second adjustment; no independent evidence supplied in abstract.

invented entities (1)

Virtual clients no independent evidence
purpose: Maintain graph topology and global embeddings on the central server after user removal without recovering or exposing the removed entity's data.
New construct introduced to solve the topology-preservation problem in GFL unlearning.

pith-pipeline@v0.9.0 · 5538 in / 1378 out tokens · 59896 ms · 2026-05-08T18:44:01.176183+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

61 extracted references · 3 canonical work pages · 1 internal anchor

[1]

IEEE Symposium on Security and Privacy , pages=

Towards making systems forget with machine unlearning , author=. IEEE Symposium on Security and Privacy , pages=
[2]

Advances in Neural Information Processing Systems , volume=

Revisiting heterophily for graph neural networks , author=. Advances in Neural Information Processing Systems , volume=
[3]

Nature Reviews Methods Primers , volume=

Graph neural networks , author=. Nature Reviews Methods Primers , volume=. 2024 , pages=

2024
[4]

IEEE Transactions on Neural Networks and Learning Systems , volume=

A comprehensive survey on graph neural networks , author=. IEEE Transactions on Neural Networks and Learning Systems , volume=. 2020 , publisher=

2020
[5]

2025 , pages=

Li, Kemou and Wang, Qizhou and Wang, Yue and Li, Fengpeng and Liu, Jun and Han, Bo and Zhou, Jiantao , journal=. 2025 , pages=

2025
[6]

Nature Machine Intelligence , pages=

Rethinking machine unlearning for large language models , author=. Nature Machine Intelligence , pages=
[7]

International Conference on Machine Learning , year=

GRU: Mitigating the Trade-off between Unlearning and Retention for Large Language Models , author=. International Conference on Machine Learning , year=
[8]

Advances in Neural Information Processing Systems , volume=

Attention is all you need , author=. Advances in Neural Information Processing Systems , volume=. 2017 , pages=

2017
[9]

Advances in Neural Information Processing Systems , year=

Gradient episodic memory for continual learning , author=. Advances in Neural Information Processing Systems , year=
[10]

International Conference on Machine Learning , pages =

Certified data removal from machine learning models , author =. International Conference on Machine Learning , pages =
[11]

Golatkar, Aditya and Achille, Alessandro and Soatto, Stefano , booktitle=
[12]

ACM SIGSAC Conference on Computer and Communications Security , pages=

Deep learning with differential privacy , author=. ACM SIGSAC Conference on Computer and Communications Security , pages=
[13]

Computers & Industrial Engineering , volume=

A review of applications in federated learning , author=. Computers & Industrial Engineering , volume=
[14]

Journal of Industrial Engineering and Applied Science , volume=

Anomaly pattern detection in high-frequency trading using graph neural networks , author=. Journal of Industrial Engineering and Applied Science , volume=
[15]

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Cho, Kyunghyun and Van Merri. arXiv preprint arXiv:1406.1078 , year=

work page internal anchor Pith review arXiv
[16]

2023 , pages=

Fan, Chongyu and Liu, Jiancheng and Zhang, Yihua and Wong, Eric and Wei, Dennis and Liu, Sijia , journal=. 2023 , pages=

2023
[17]

Advances in Neural Information Processing Systems , pages=

Unified gradient-based machine unlearning with remain geometry enhancement , author=. Advances in Neural Information Processing Systems , pages=
[18]

ACM SIGSAC Conference on Computer and Communications Security , pages=

When machine unlearning jeopardizes privacy , author=. ACM SIGSAC Conference on Computer and Communications Security , pages=
[19]

IEEE Transactions on Neural Networks and Learning Systems , volume=

Fast yet effective machine unlearning , author=. IEEE Transactions on Neural Networks and Learning Systems , volume=
[20]

IEEE Symposium on Security and Privacy , pages=

Machine unlearning , author=. IEEE Symposium on Security and Privacy , pages=
[21]

Dave, Ayush and Agrawal, Animesh Kumar , booktitle=
[22]

de Oliveira, J. A prote. NATIVA-Revista de Ci
[23]

2022 , pages=

Bonta, Rob , journal=. 2022 , pages=

2022
[24]

Annual Computer Security Applications Conference , pages=

Model inversion attacks against collaborative inference , author=. Annual Computer Security Applications Conference , pages=
[25]

Liu, Bo and Ding, Ming and Shaham, Sina and Rahayu, Wenny and Farokhi, Farhad and Lin, Zihuai , journal=
[26]

Ko, Hyeyoung and Lee, Suyeon and Park, Yoonseo and Choi, Anna , journal=
[27]

Advances in Neural Information Processing Systems , volume=

How powerful are k-hop message passing graph neural networks , author=. Advances in Neural Information Processing Systems , volume=
[28]

arXiv preprint arXiv:2302.05631 , year=

A survey on spectral graph neural networks , author=. arXiv preprint arXiv:2302.05631 , year=

work page arXiv
[29]

2025 , pages=

Zhongtian Ma and Qiaosheng Zhang and Bocheng Zhou and Yexin Zhang and Shuyue Hu and Zhen Wang , booktitle=. 2025 , pages=

2025
[30]

Tesfay, Welderufael B and Hofmann, Peter and Nakamura, Toru and Kiyomoto, Shinsaku and Serna, Jetzabel , booktitle=
[31]

Mathematics , VOLUME =

Hu, Kai and Wu, Jiasheng and Li, Yaogen and Lu, Meixia and Weng, Liguo and Xia, Min , TITLE =. Mathematics , VOLUME =. 2022 , NUMBER =

2022
[32]

Tan, Yue and Liu, Yixin and Long, Guodong and Jiang, Jing and Lu, Qinghua and Zhang, Chengqi , booktitle=
[33]

and Farokhi, Farhad and Jin, Shi and Quek, Tony Q

Wei, Kang and Li, Jun and Ding, Ming and Ma, Chuan and Yang, Howard H. and Farokhi, Farhad and Jin, Shi and Quek, Tony Q. S. and Vincent Poor, H. , journal=. 2020 , volume=

2020
[34]

Fu, Xingbo and Zhang, Binchi and Dong, Yushun and Chen, Chen and Li, Jundong , journal=
[35]

Yang, Qiang and Liu, Yang and Chen, Tianjian and Tong, Yongxin , journal=
[36]

Liu, Ziyao and Jiang, Yu and Shen, Jiyuan and Peng, Minyi and Lam, Kwok-Yan and Yuan, Xingliang and Liu, Xiaoning , journal=
[37]

A Practical Guide, 1st Ed., Cham: Springer International Publishing , volume=

The eu general data protection regulation (gdpr) , author=. A Practical Guide, 1st Ed., Cham: Springer International Publishing , volume=
[38]

2024 , pages=

Zhang, Ruiqi and Lin, Licong and Bai, Yu and Mei, Song , journal=. 2024 , pages=

2024
[39]

2025 , pages=

Qizhou Wang and Jin Peng Zhou and Zhanke Zhou and Saebyeol Shin and Bo Han and Kilian Q Weinberger , booktitle=. 2025 , pages=

2025
[40]

Kipf et al.Variational Graph Auto- Encoders

Variational graph auto-encoders , author=. arXiv preprint arXiv:1611.07308 , year=

work page arXiv
[41]

ACM Transactions on Intelligent Systems and Technology , volume=

Atom-motif contrastive transformer for molecular property prediction , author=. ACM Transactions on Intelligent Systems and Technology , volume=
[42]

AAAI Conference on Artificial Intelligence , pages=

Modeling inter-intra heterogeneity for graph federated learning , author=. AAAI Conference on Artificial Intelligence , pages=
[43]

International Conference on Data Mining , year=

Traffic pattern sharing for federated traffic flow prediction with personalization , author=. International Conference on Data Mining , year=
[44]

Hyperspectral image classification with contrastive graph convolutional network , year=

Yu, Wentao and Wan, Sheng and Li, Guangyu and Yang, Jian and Gong, Chen , journal=. Hyperspectral image classification with contrastive graph convolutional network , year=
[45]

IEEE Transactions on Cybernetics , volume=

Two-stream spatial--temporal graph convolutional networks for driver drowsiness detection , author=. IEEE Transactions on Cybernetics , volume=
[46]

International Conference on Learning Representations , year=

How powerful are graph neural networks? , author=. International Conference on Learning Representations , year=
[47]

International Conference on Artificial Intelligence and Statistics , pages=

Communication-efficient learning of deep networks from decentralized data , author=. International Conference on Artificial Intelligence and Statistics , pages=
[48]

2025 , url =

Yu, Wentao , journal=. 2025 , url =

2025
[49]

2025 , url =

Yu, Wentao and Gong, Chen and Han, Bo and Fan, Lixin and Yang, Qiang , journal=. 2025 , url =

2025
[50]

2025 , volume=

Zhou, Hang and Yu, Wentao and Wan, Sheng and Tong, Yongxin and Gu, Tianlong and Gong, Chen , journal=. 2025 , volume=

2025
[51]

Subgraph federated learning with missing neighbor generation , year =

Zhang, Ke and Yang, Carl and Li, Xiaoxiao and Sun, Lichao and Yiu, Siu Ming , booktitle =. Subgraph federated learning with missing neighbor generation , year =
[52]

SIAM Journal on Scientific Computing , volume=

A fast and high quality multilevel scheme for partitioning irregular graphs , author=. SIAM Journal on Scientific Computing , volume=
[53]

International Conference on Machine Learning , pages=

Sablayrolles, Alexandre and Douze, Matthijs and Schmid, Cordelia and Ollivier, Yann and J. International Conference on Machine Learning , pages=
[54]

2022 , pages=

Halimi, Anisa and Kadhe, Swanand and Rawat, Ambrish and Baracaldo, Nathalie , journal=. 2022 , pages=

2022
[55]

2022 , volume=

Wu, Leijie and Guo, Song and Wang, Junxiao and Hong, Zicong and Zhang, Jie and Ding, Yaohong , journal=. 2022 , volume=

2022
[56]

Chourasia, Rishav and Shah, Neil , booktitle=
[57]

Nature Communications , year=

A federated graph neural network framework for privacy-preserving personalization , author=. Nature Communications , year=
[58]

ACM on Web Conference , year=

Subgraph federated unlearning , author=. ACM on Web Conference , year=
[59]

International Conference on Machine Learning , pages=

Revisiting semi-supervised learning with graph embeddings , author=. International Conference on Machine Learning , pages=
[60]

2023 , pages =

Platonov, Oleg and Kuznedelev, Denis and Diskin, Michael and Babenko, Artem and Prokhorenkova, Liudmila , booktitle=. 2023 , pages =

2023
[61]

International Conference on Computational Statistics , year=

Large-scale machine learning with stochastic gradient descent , author=. International Conference on Computational Statistics , year=