{"id":"f6e22bcd-08da-4897-8204-6f806f99b1d9","arxiv_id":"2505.09702","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FGU is a shard-based graph unlearning framework with a local fairness regularizer plus a global disparity alignment step, reporting lower demographic parity and equal opportunity gaps than existing graph unlearning baselines.","lead":"A new graph unlearning method, FGU, adds a two-level fairness correction: each shard model is retrained with a demographic parity regularizer, then the shard models are aligned to reduce global disparity. The paper claims standard graph unlearning methods increase demographic parity and equal opportunity gaps after deletions, and that FGU reduces those gaps while keeping accuracy and privacy close to retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Global alignment debiases a parameter-averaged model, but deployed inference averages shard posteriors; without evidence the debiasing transfers, the reported fairness improvement may not come from the global module.","rationale":"The reader's weakest assumption identifies exactly the mismatch I consider most load-bearing: the global objective in Eqs. (8)-(9) uses θ~ = Σ_k λ_k θ_k, while the deployed predictor in Section III-A averages posterior vectors. For nonlinear GNNs these are different predictors, and no argument or experiment bridges them. This concern directly targets the novelty claim (bi-level debiasing, with global alignment correcting aggregation-time bias) rather than mere reproducibility issues like the α/β inconsistency or missing code. The hyperparameter inconsistency (Section IV-F says α=3.0, β=1.5; Appendix I says α=0.5, β=1) and the absence of released code are real but secondary; they affect reproducibility of the exact tables, not whether the mechanism works as claimed. The motivating 'unlearning introduces bias' observation is also confounded by distribution shift, but the paper's core comparison to retraining on remaining data partly controls for that, so I do not treat it as the central threat. A concrete implementation-level check can settle the aggregation mismatch: evaluate both parameter-averaged and posterior-averaged predictions, and ablate L_global. If the global term only improves the parameter-averaged model, the central claim should be downgraded to a local-debiasing result. Since the reader already assigned CONDITIONAL with this same concern, my stress-test does not change the verdict; it reinforces the condition.","tokens_in":24356,"tokens_out":3913,"duration_ms":43903,"concrete_test":"Rerun the FGU experiments on at least German, Credit, and Pokec-n with the exact training procedure, then evaluate fairness under both aggregations: (a) parameter-average prediction ŷ = argmax f_{Σ_k λ_k θ_k}(A,X) and (b) posterior-average prediction ŷ = argmax Σ_k λ_k softmax(f_{θ_k}(A,X)) (with the same graph sent to each shard as in Section III-A). Report ΔDP and ΔEO for both. Then run the same protocol with L_global removed (local-only FGU). If posterior-averaged ΔDP/ΔEO for full FGU is not substantially lower than the local-only variant, or if posterior-averaged gaps are materially higher than parameter-averaged gaps (e.g., more than 2×), the global alignment claim is unsupported and the reported fairness gains should be attributed to local debiasing rather than bi-level debiasing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FGU achieves superior fairness rests on the bi-level debiasing, and specifically on the global alignment term. In Section III-C, the global fairness loss is defined on the parameter-averaged model: θ~ = Σ_k λ_k θ_k and F_global = |P(M_θ~(G')=1|S'=0) − P(M_θ~(G')=1|S'=1)| (Eqs. 5, 8, 9). This is the objective whose gradient updates both λ and the shard parameters. However, Section III-A (\"Aggregation and Inference\") states that at inference the final prediction is the weighted sum of the shard models' posterior vectors, not the posterior of the parameter-averaged model. For a nonlinear GNN, Σ_k λ_k softmax(f_{θ_k}(x)) ≠ softmax(f_{Σ_k λ_k θ_k}(x)) in general, and the paper provides no bound, monotonicity argument, or ablation showing that reducing the DP gap of the parameter-averaged predictor reduces the DP gap of the posterior-averaged predictor used for evaluation. If the reported ΔDP/ΔEO numbers are computed with posterior averaging, then the global alignment term may be optimizing an objective that is different from the deployed predictor, and the observed fairness could be attributable to the shard-level regularizer F_k alone, or to tuning λ against the evaluation metric. The ablation in Table IV does not resolve this: it compares local-only and global-only variants without reporting both aggregation rules, so it cannot distinguish whether the global loss transfers to posterior averaging. This is load-bearing because the novelty of FGU over simply debiasing each shard is precisely the claim that global alignment fixes bias introduced during aggregation; if that transfer fails, the method's contribution collapses to local regularizers plus weighted averaging.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies whether graph unlearning introduces group bias and proposes FGU, a shard-based exact graph unlearning method with two debiasing levels: a local regularizer applied when retraining each shard, and a global alignment term that penalizes demographic parity gaps of a parameter-averaged shard ensemble. The authors report on six datasets that FGU achieves lower ΔDP and ΔEO than graph unlearning baselines and fair retraining while maintaining utility and membership-inference privacy, and they include ablations, hyperparameter sensitivity, fairness-aware GNN comparisons, and a link prediction extension.","tokens_in":24615,"tokens_out":9053,"duration_ms":82711,"significance":"If the claims are substantiated, FGU would be a practical contribution to graph unlearning with group fairness, with efficiency advantages from sharding. The paper is broad in scope: six datasets, multiple unlearning ratios and request types, privacy attacks, and comparisons to fairness-aware GNNs. The experiments are extensive and the idea of bi-level debiasing is intuitive. However, the central mechanism of the global alignment step is not validated as described, and the hyperparameter reporting is contradictory, so the current version does not yet establish that the global module provides the claimed benefit.","major_comments":[{"comment":"The global alignment loss is computed on the parameter-averaged model θ~ = Σ_k λ_k θ_k (Eqs. 8-9), while the deployed inference described in Section III-A averages the shard models' posterior vectors. For a nonlinear GCN, Σ_k λ_k softmax(f_{θ_k}(x)) differs from softmax(f_{Σ_k λ_k θ_k}(x)), so reducing the ΔDP of the parameter-averaged model need not reduce the ΔDP of the posterior-averaged model used in the reported numbers. The paper provides no bound, monotonicity argument, or ablation that evaluates both aggregation rules. As a result, the observed fairness improvements may be entirely attributable to the local regularizer F_k or to tuning λ on the evaluation metric. Please add an analysis or experiment that either justifies the transfer or defines F_global directly on the posterior-averaged predictor used at inference.","section":"Section III-C vs Section III-A"},{"comment":"The manuscript reports conflicting values for the fairness regularization weights. Appendix I states 'ultimately selecting α = 0.5 and β = 1 for the experiments,' whereas Section IV-F states 'we choose α = 3.0 and β = 1.5 to perform FGU.' In addition, Section IV-F says α and β are varied over {0.5, 1.5, 3.0, 5.0, 7.0} but the Figure 5 axes enumerate {0.001, 0.01, 0.1, 0.5, 1, 5, 10}. Since the accuracy-fairness trade-off is highly sensitive to these weights (Figure 5), please state unambiguously which values produced Table I and the other main results, and re-run or justify the reported numbers under the correct hyperparameters.","section":"Section IV-F and Appendix I"},{"comment":"The ablation does not support the stated conclusion that FGU is better than its two variants. In Table IV, Local Debiasing alone attains ACC 77.1 ± 2.2 and F1 89.8 ± 0.4 with ΔDP 3.1 ± 1.6 and ΔEO 3.2 ± 2.9, while full FGU attains ACC 66.8 ± 1.1, F1 77.1 ± 2.1, ΔDP 2.8 ± 0.2, and ΔEO 2.9 ± 0.3. Thus the global alignment term reduces the fairness gaps by only about 0.3 points while lowering accuracy by more than 10 points. This is the opposite of a favorable fairness-utility trade-off and weakens the claim that bi-level debiasing is beneficial. Also, the second ΔEO row in Table IV should presumably be labeled ΔDP.","section":"Table IV"},{"comment":"Because ΔDP (and to a lesser extent ΔEO) is both the training objective in F_k and F_global (Eqs. 4-5) and the evaluation metric in Table I, the fairness improvements over non-fair baselines are to a significant degree a fitted outcome. This is a common limitation of in-processing fairness methods, but the paper should acknowledge it explicitly and, ideally, report at least one additional fairness measure or a held-out sensitive attribute to support the claim of 'superior fairness' beyond the optimized objective.","section":"Section II and III"}],"minor_comments":[{"comment":"The metric names are swapped in the text: 'Demographic Parity Difference △EO' and 'Equal Opportunity Difference △DP' appear where Eq. (1) defines ΔDP and ΔEO in the opposite way.","section":"Section IV-A and Appendix I"},{"comment":"Equation (2) mentions a regularization parameter γ that does not appear in the equation, and the notation U_k(θ_k) is used before its definition in Eq. (3); please rewrite for clarity.","section":"Equation (2)"},{"comment":"The dataset description lists 'Pokec-z and Pokec-z' where the second subset should be 'Pokec-n'.","section":"Appendix C"},{"comment":"The runtime bullet reads 'FGU is consistently less than 2.1 seconds and 5.2 seconds on German and seconds on Bail'; the sentence is incomplete, and the name 'Amenisac' is not defined in the manuscript.","section":"Section IV-I"},{"comment":"The stated percentages for FGU's ΔDP relative to GEditor, GEraser, GDelete, and GIF (5%, 6.0%, 5.8%, and 4.7%) do not match the German row in Table I; for example, 1.7/27.1 is about 6.3%, not 5%.","section":"Section IV-B, observation 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a solid empirical scope, but the global alignment mechanism and the reported hyperparameters are currently not sufficiently established. I recommend asking the authors to either change the method so that F_global is defined on the posterior-averaged predictor used at inference, or provide a targeted experiment demonstrating transfer from the parameter-averaged objective to the deployed predictor. They should also resolve the α/β inconsistency before the next round."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2505.09702. First, it is a genuinely new combination: group fairness constraints inside a shard-based exact graph unlearning framework, with a local regularizer on each shard and a global alignment term that tries to reduce demographic parity after aggregation. The experimental surface is broad — six datasets, node and edge deletion at several ratios, membership inference attacks, an ablation, a link-prediction appendix, and comparisons against fairness-aware GNNs. Second, there is a load-bearing mismatch between what the method optimizes and what it deploys. Global alignment is defined on the parameter-averaged model θ~ = Σλ_k θ_k (Eqs. 8–9), but Section III-A says inference averages the shard models' posterior vectors. For a nonlinear GNN those are different predictors, and the paper gives no bound, monotonicity argument, or ablation showing that debiasing the parameter-averaged model transfers to the posterior-averaged model used to report ΔDP and ΔEO. The ablation in Table IV does not settle it, because it does not report both aggregation rules. This is not a nitpick: the global module is the main novelty over simply debiasing each shard, and if the transfer fails, the reported fairness gains could come from the local regularizer alone or from tuning λ against the evaluation metric.\n\nThe other soft spots are smaller but real. The hyperparameters are reported inconsistently: Section IV-A says α=0.5, β=1, while Section IV-F says α=3.0, β=1.5, and the search range in Figure 5 ({0.5,1.5,3.0,5.0,7.0}) conflicts with the range in Appendix I ({0.001,...,10}). Table II shows FGU's membership-inference attack accuracy exactly matching Retrain to one decimal across all datasets and ratios; that looks like a reporting artifact and needs verification. The motivating claim that unlearning introduces bias is overstated: non-uniform deletion shifts group base rates, so ΔDP can rise even for a perfectly calibrated retrained model, and the paper does not decompose distribution shift from classifier discrimination.\n\nWhat is good: the shard-level fairness regularizer is simple and sensible, the privacy story (FGU near random-guess attack accuracy) is plausible for exact unlearning, and the idea of aligning shard models to fix aggregation-level bias is worth taking seriously even if this instantiation is not fully pinned down. The paper deserves a serious referee, but the referee should demand a clear statement of the inference rule, an ablation that varies the aggregation method, a fix to the hyperparameter reporting, and code or detailed configs. Without those, the central fairness claim is not reproducible.\n\nMy read: conditional accept at best. The core idea is sound, the current evidence does not yet support the headline that FGU's global alignment is what delivers the fairness gains. I would not cite it in my own work until the mismatch is resolved, but I would send it to review and ask for a careful revision.","headline":"Useful first pass at fair graph unlearning, but the global debiasing objective and the deployed aggregation may be optimizing different predictors; needs a revision that addresses this before the fairness claims can be trusted.","tokens_in":25271,"tokens_out":1922,"would_cite":false,"duration_ms":21555,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Graph unlearning introduces bias; a bi-level debiasing method removes it while preserving privacy and accuracy.","keywords":["graph unlearning","group fairness","demographic parity","equal opportunity","graph neural networks","shard-based unlearning","membership inference","node classification"],"falsifier":"Run FGU on a fixed unlearning request with the global-alignment term disabled but the learned $\\lambda$ kept, and compare $\\Delta_{\\mathrm{DP}}$ of the deployed posterior-averaged predictor; if the gap stays essentially unchanged, the global alignment is not carrying the reported fairness improvement. A complementary check is to evaluate $\\Delta_{\\mathrm{DP}}$ for predictions made from the parameter-averaged weights versus from the posterior average on the same nodes; a large discrepancy would show the objective is optimizing a predictor different from the one users see.","tokens_in":24059,"feed_emoji":"⚖️","tokens_out":6465,"duration_ms":60472,"temperature":0.7,"pith_summary":"Deleting a user's node or edge from a trained graph model can quietly change how the model treats protected groups, so the very act of complying with a deletion request can make otherwise acceptable predictions unfair. The paper establishes this empirically and then claims a fix: FGU, a graph unlearning method that first debiases each shard model locally and then aligns all shard models against a global fairness objective. The intended payoff is that after unlearning, demographic parity and equal opportunity gaps drop below those of ordinary retraining, accuracy stays near retraining levels, and membership-inference attacks perform no better than random guessing. In short, the paper aims to show that forgetting and fairness are compatible, at a computational cost far below full retraining.","feed_headline":"Bias creeps back after graph unlearning; FGU erases both","feed_subtitle":"Two-level debiasing keeps parity gaps near fair retraining while membership attacks stay at chance.","key_machinery":"The load-bearing mechanism is the bi-level debiasing objective over a sharded model. FGU partitions the graph into $K$ shards, trains a shard model $\\theta_k$ on each, and aggregates predictions by the weighted sum of shard posteriors with learned importance weights $\\lambda_k$; for the fairness objective it instead aggregates parameter weights as $\\tilde{\\theta} = \\sum_k \\lambda_k \\theta_k$. Shard-level debiasing adds a local demographic-parity penalty $F_k$ to each shard's retraining loss, while global alignment minimizes $L_{\\mathrm{global}} = U_{\\mathrm{global}} + \\alpha F_{\\mathrm{global}}$, where $F_{\\mathrm{global}}$ measures the demographic-parity gap of the aggregated model over the whole remaining graph. The two losses are coupled through the shard objective $L_k = U_k + \\alpha_k F_k + \\beta_k L_{\\mathrm{global}}$, and an alternating optimization updates shard weights every epoch and importance weights every $t_1$ epochs. This lets fairness be enforced at the level where bias actually enters, the message-passing aggregation of shard predictions, without retraining the entire dataset.","core_discovery":"The paper's central claim is that standard graph unlearning methods systematically introduce bias: after nodes or edges are removed, the post-unlearning model's predictions become strongly correlated with the sensitive attribute, and the gap grows with the deletion ratio and is worse when deletions come from the unprivileged group. FGU counters this with bi-level debiasing: within each shard it retrains with a demographic-parity regularizer, and across shards it aligns the aggregated model by a global loss that penalizes disparity in prediction rates between sensitive groups. On six datasets and across node and edge deletion requests, FGU reports $\\Delta_{\\mathrm{DP}}$ and $\\Delta_{\\mathrm{EO}}$ values below both retraining and all graph-unlearning baselines, accuracy and F1 comparable to fair retraining, membership-inference attack accuracy near 50%, and a better accuracy-fairness trade-off than fairness-aware GNN baselines trained directly on the remaining data. The paper positions FGU as the first graph unlearning framework that simultaneously preserves privacy of deleted data and fairness of the post-unlearning model.","pith_inferences":["A natural next experiment is to compute the global-alignment loss on the posterior-averaged predictor instead of the parameter-averaged one; if results match, the mechanism transfers, and if not, the simpler shard regularizer is the active ingredient.","The paper's observation that unlearning from the unprivileged group induces more bias suggests a testable corollary: bias introduced by deletion should scale with the level of homophily in the graph and with the concentration of deletion in one sensitive group, which could predict where fair unlearning is hardest.","The same bi-level design could be applied to federated graph learning, where clients are natural shards and global alignment plays the role of server-side aggregation, extending fair unlearning to settings where data cannot leave its owner."],"forward_implications":["Unlearning requests are not fairness-neutral: deleting nodes or edges shifts prediction rates across sensitive groups, so any deployment of the right to be forgotten on graph data should audit fairness before and after deletion.","Because FGU's fairness gains come from shard-level retraining plus global alignment on the remaining graph, the method needs no access to the original full dataset after partitioning, which keeps unlearning efficient.","If FGU is correct, fairness and deletion privacy do not trade off against each other: membership-inference attack accuracy stays at chance while demographic parity and equal opportunity gaps remain near fair-retraining levels.","The accuracy-fairness comparison against fairness-aware GNN baselines implies that debiasing before or during unlearning, rather than after, is the right intervention point for post-deletion models."],"supporting_citations":[{"why":"supplies the shard-training-and-retraining scheme FGU adapts for graph data.","marker":"[29]"},{"why":"defines the exact graph unlearning baseline and the importance-weighted shard aggregation FGU builds on.","marker":"[12]"},{"why":"documents that machine unlearning on tabular data introduces disparity, the prior result FGU extends to graphs.","marker":"[33]"},{"why":"gives an influence-function graph unlearning baseline whose bias FGU measures and improves upon.","marker":"[23]"},{"why":"provides an approximate graph unlearning baseline for comparison on privacy and fairness.","marker":"[13]"},{"why":"defines demographic parity, the fairness notion used in the shard regularizer.","marker":"[17]"},{"why":"defines equal opportunity, the second fairness metric reported for FGU.","marker":"[25]"},{"why":"is a fairness-aware GNN baseline used in the accuracy-fairness trade-off comparison.","marker":"[41]"},{"why":"is a data-editing debiasing baseline used in the trade-off comparison.","marker":"[42]"}],"fun_headline_variants":["Unlearning introduces bias; FGU removes it with two-level debias","Graph unlearning amplifies bias; FGU balances across groups","Deleting data skews models; FGU's bi-level debiasing fixes it","FGU: fair unlearning without the bias leak"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fairness adjustment is computed on an average of the models' internal weights, while the predictions users actually see come from averaging the models' probability outputs; for a deep graph network these are different predictors, and the paper does not show that making one fair makes the other fair.","fun_headline_variants_meta":{"raw":{"variants":["Unlearning introduces bias; FGU removes it with two-level debias","Graph unlearning amplifies bias; FGU balances across groups","Deleting data skews models; FGU's bi-level debiasing fixes it","FGU: fair unlearning without the bias leak"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000356,"raw_usage":{"total_tokens":1967,"prompt_tokens":1018,"completion_tokens":949,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":872}},"tokens_in":634,"tokens_out":949,"duration_ms":6670,"temperature":1.0,"reasoning_tokens":872,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:26:44.656538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FGU on a fixed unlearning request with the global-alignment term disabled but the learned $\\lambda$ kept, and compare $\\Delta_{\\mathrm{DP}}$ of the deployed posterior-averaged predictor; if the gap stays essentially unchanged, the global alignment is not carrying the reported fairness improvement. A complementary check is to evaluate $\\Delta_{\\mathrm{DP}}$ for predictions made from the parameter-averaged weights versus from the posterior average on the same nodes; a large discrepancy would show the objective is optimizing a predictor different from the one users see.","supporting_citations":[{"cited_title":"Machine unlearning,","cited_arxiv_id":null,"evidence_quote":"supplies the shard-training-and-retraining scheme FGU adapts for graph data."},{"cited_title":"Graph unlearning,","cited_arxiv_id":null,"evidence_quote":"defines the exact graph unlearning baseline and the importance-weighted shard aggregation FGU builds on."},{"cited_title":"Fair machine unlearning: Data removal while mitigating disparities,","cited_arxiv_id":null,"evidence_quote":"documents that machine unlearning on tabular data introduces disparity, the prior result FGU extends to graphs."},{"cited_title":"Gif: A general graph unlearning strategy via influence function,","cited_arxiv_id":null,"evidence_quote":"gives an influence-function graph unlearning baseline whose bias FGU measures and improves upon."},{"cited_title":"Fairness through awareness,","cited_arxiv_id":null,"evidence_quote":"defines demographic parity, the fairness notion used in the shard regularizer."},{"cited_title":"Equality of oppor- tunity in supervised learning,","cited_arxiv_id":null,"evidence_quote":"defines equal opportunity, the second fairness metric reported for FGU."},{"cited_title":"Say no to the discrimination: Learning fair graph neural networks with limited sen- sitive attribute information,","cited_arxiv_id":null,"evidence_quote":"is a fairness-aware GNN baseline used in the accuracy-fairness trade-off comparison."},{"cited_title":"Edits: Modeling and mitigating data bias for graph neural networks,","cited_arxiv_id":null,"evidence_quote":"is a data-editing debiasing baseline used in the trade-off comparison."}],"review_version":1}