REVIEW 3 major objections 5 minor 29 references
Achieving Hilbert-Schmidt Independence Under R\'enyi Differential Privacy for Fair and Private Data Generation
T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read FLIP generates heterogeneous tabular data whose non-protected features are statistically independent of the protected attribute, under Rényi differential privacy.
desk verdict FLIP is a serious empirical attempt at task-agnostic fairness under DP that deserves a real referee, but the CKA-based loss does not actually establish the claimed X⊥S independence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Centered Kernel Alignment (CKA), a normalized Hilbert-Schmidt Independence Criterion between kernel matrices, computed on transposed activation matrices of the two protected groups; negative CKA is the disentanglement term in Eq. 10. The other load-bearing pieces are the sliced Wasserstein divergence penalty that anchors the fair model to the Phase-1 representation, and the balanced Poisson sampling scheme under RDP: group-wise sample rates (Eq. 12), the monotonicity Proposition 1, and the global noise multiplier (Eq. 13).
What would settle it
On a dataset with a strong known correlation between a non-protected feature and the protected attribute, generate data with FLIP at λ=4 and ε=3, then run a statistical independence test (e.g., Hilbert-Schmidt independence criterion with a Gaussian kernel) between the generated non-protected features and the protected attribute. If the test rejects independence with high power, or if a simple classifier recovers the protected attribute from the generated features, the disentangled-fairness claim fails for that setting.
Extended reading notes
Core claim
FLIP's central claim is that a two-phase training pipeline can disentangle a binary protected attribute from the rest of heterogeneous tabular data while preserving Rényi differential privacy. Phase 1 trains a transformer-based β-VAE for reconstruction quality; Phase 2 adds a fairness objective that penalizes negative CKA between protected-group activation matrices at the latent, detokenizer, and decoder stages, while a divergence penalty (sliced Wasserstein distance, or the ELBO at the detokenizer) keeps the perturbed model near the Phase-1 reference. Balanced Poisson sampling gives each protected group its own sample rate and noise level, with Proposition 1 showing the RDP guarantee is con
Load-bearing premise
The load-bearing premise is that pushing the two protected groups' neuron activation patterns to resemble each other—through negative CKA at three stages—is enough to make the generated features truly independent of the protected attribute; the paper gives evidence but no proof of that sufficiency.
Editorial extensions
If this is right
- Synthetic data from FLIP can be handed to unknown downstream tasks without inheriting the protected attribute's bias, because decoupling is defined on the data rather than on a target label.
- Other generative models can adopt group-wise sampling rates under RDP: Proposition 1 guarantees that the privacy loss is bounded by the largest group-wise sample rate, and Eq. 13 composes the group-specific noise multipliers.
- At a fixed privacy budget, stronger fairness regularization is required to reach the same disentanglement as without DP, so privacy and fairness budgets should be tuned jointly rather than separately.
- The fairness-quality trade-off is explicit and gradual: increasing λ monotonically lowers fidelity and diversity metrics, and the paper interprets this as the cost of removing real bias from the learned distribution.
Reading between the lines
- Editorial inference: The CKA objective aligns covariance patterns, not full statistical independence; the paper does not prove that minimizing negative CKA at the three stages implies X ⊥ S as Definition 3 requires.
- Editorial inference: A-NCB and BER are strong proxies, but neither certifies disentangled fairness; a direct hypothesis test of independence on the generated samples would settle whether the title-level claim holds.
- Editorial inference: The task-agnostic framing implies that quality should be measured against an unbiased reference distribution, not the original biased data; without such a reference, reported quality degradation mixes genuine bias removal with utility loss.
- Editorial inference: The same latent-intervention machinery could be extended to multiple protected attributes or continuous protected features by replacing binary group splits with a kernel-based independence penalty, though the paper does not explore this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FLIP, a transformer-based variational autoencoder with latent diffusion for generating heterogeneous tabular data. FLIP is designed to provide task-agnostic fairness—formalized as "disentangled fairness" (Definition 3: X̃ ⊥ S̃)—while guaranteeing Rényi differential privacy. Training is split into two phases: Phase 1 learns a quality-focused latent representation; Phase 2 applies a fairness intervention that aligns neuron activation patterns across protected groups using negative centered kernel alignment (CKA), together with balanced sampling and a uniform protected-attribute loss. Privacy is provided by DP-SGD with an RDP accountant, extended to group-wise balanced Poisson sampling. Experiments on Adult, ACS-I (AL, UT), and Dutch report large improvements in balanced error rate and adversarial normalized cluster balance relative to baselines, at a moderate AUC cost, under ε=3 RDP.
Significance. The intended contribution is significant. A task-agnostic notion of fairness for synthetic data, combined with formal RDP guarantees, addresses a real gap in the literature, which largely evaluates fairness relative to a predefined downstream task. The paper also makes a methodological contribution by formulating fairness as a data-level property and by using adversarial evaluation (A-NCB) to avoid spurious fairness from random protected-attribute assignment. The empirical claims, if fully supported, would show that FLIP substantially outperforms existing fair/private tabular generators on BER and A-NCB. The manuscript is generally clear about its task-agnostic framing and includes a useful limitation discussion. However, as detailed below, the central theoretical link between the CKA-based training objective and the formal independence condition in Definition 3 is not established, and the privacy accounting for group-wise sampling needs scrutiny.
major comments (3)
- [§4.2.2, Eqs. (5)–(7), Eq. (10)] The paper claims that minimizing negative CKA between protected-group activation matrices 'encourages statistical independence between latent representations and the protected feature' and ultimately achieves Definition 3 (X̃ ⊥ S̃), but no derivation is given. With a linear kernel, CKAT(A,B) = ‖ATB‖_F² / (‖ATA‖_F ‖BTB‖_F) is computed after centering with H (Eqs. 5–7). Centering removes column means, so CKAT is invariant to group-specific mean differences. Consequently, if P(X|S=0) and P(X|S=1) differ only by a shift in the first moment, the disentanglement gradient is exactly zero, and Eq. (4), the paper's own equivalent condition for X⊥S, is not approached. HSIC with a linear kernel measures covariance similarity, not statistical independence; even in the population limit, HSIC=0 implies independence only for characteristic kernels. No component of Lfair (Eq. 10) or LS (Eq. 3) is shown
- [§4.4, Proposition 1, Eq. (13)] The privacy accounting for balanced group-wise sampling is not fully justified. Proposition 1 shows monotonicity of a scalar RDP bound in the sampling rate γ, i.e., if ε(α, γ) ≤ γ²·2α/σ², then a larger γ gives a larger bound. But this does not by itself account for the actual mechanism used here, where different protected groups are sampled at different rates γ(s) in each iteration and the RDP loss is composed over L iterations per epoch. The paper asserts that using γmax = max_s γ(s) is sufficient in the worst case, but the composition of heterogeneous per-record sampling probabilities is not identical to running uniform sampling at γmax in every step, particularly when group membership is known and balanced batches are enforced without replacement. Equation (13) for deriving σglobal from per-group noise multipliers is taken from IDP-SGD; the paper should either provide a self-contained
- [§6.1, §6.4, Table 3] The hyperparameter λ for FLIP is selected using the same task-agnostic fairness metrics (BER, A-NCB) on the same datasets, as discussed in §6.1 ('at a λ value of 4, both BER and A-NCB plateau'), and then Table 3 reports FLIP λ=4 as the headline comparison on those very metrics. This introduces a circularity burden: the reported +67.84% BER and +20.87% A-NCB may partly reflect test-set model selection rather than a general advantage of the method. The paper should report results across the full λ grid for all baselines, or use an internal validation split for selecting λ and reserve the test folds for final comparison. Moreover, Table 3 reports only mean percentage changes without confidence intervals or error bars, despite the use of 3-fold cross-validation; given the large differences claimed, per-fold results or standard deviations would substantially strengthen the empirical claim.
minor comments (5)
- [Definition 1] The definition of ϵ-fairness says 'for any classification model f' but then requires BER(f(X̃), S̃) > ϵ. As written, a trivial random classifier with BER close to 0.5 would make the condition vacuous for reasonable ϵ. Please clarify whether f is required to be an optimal or adversarial classifier, and tie the threshold to the theoretical maximum of 0.5.
- [Eq. (3)] The softmax notation is imprecise: softmax(X_i)_j = exp(X_ij) / Σ_{k=1}^p exp(X_ik) uses p but p is not defined in the surrounding text; it should be the number of protected-attribute classes, likely |S|.
- [Eq. (13)] The derivation of σglobal is only referenced to Boenisch et al. For a journal paper, please provide at least a sketch of the derivation or state the assumptions under which the harmonic mean of per-group σ_p yields the correct global noise multiplier for DP-SGD.
- [Figure 3] The figure is extremely dense and difficult to read, especially the heatmap rows and the legend. A table of the underlying values, or splitting the figure by metric, would improve readability.
- [References] Reference 'Poission subsampled rényi differential privacy' contains a typo: 'Poisson' and 'Rényi' should be capitalized/corrected. Also, the section heading before Figure 4 appears garbled in the manuscript text.
Circularity Check
No construction-level circularity; only mild evaluation-metric tuning of the fairness weight λ. The formal independence claim is a sufficiency gap, not a circular reduction.
-
fitted input called prediction
[Section 6.1 (Figure 3) and Table 3]
"In Figure 3, we observe that at a λ value of 4, both BER and A-NCB plateaus, suggesting that further bias-mitigating perturbation is unnecessary... λ = 4 could constitute a reasonable hyperparameter for bias mitigation while minimizing the distributional change. Table 3: FLIP λ = 4, ε = 3 +67.84% +20.87%."
The headline task-agnostic fairness gains are reported at λ=4, the same value selected by inspecting BER and A-NCB in Figure 3 on the experimental data. Thus the evaluation metrics are used to choose the model configuration whose results are then reported, a mild form of fitting the evaluation target. It is not a full reduction because the training objective (negative CKA, L_S, reconstruction) is distinct from BER/A-NCB, and λ=4 is then applied uniformly across datasets; however, the reported improvements are partly selected rather than purely predicted.
full rationale
No construction-level circularity is present. The RDP privacy guarantee is imported from published composition bounds (Mironov et al., 2019; Wang et al., 2019) and the group-wise noise formula from Boenisch et al. (2023), none of which are self-citations or restatements of the paper's target result. The fairness objective — negative CKA between group activation matrices — is a distinct quantity from the evaluation metrics BER and A-NCB; the paper does not directly optimize BER or A-NCB in the loss. The λ selection is a mild hyperparameter tuning on the evaluation metric, which we flag but score as minor. The deeper concern that linear-kernel CKA, being invariant to group mean differences, does not by construction yield X̃⊥⊥S̃ is a correctness/sufficiency gap rather than a circular equation-to-equation reduction; per the hard rules it does not raise the circularity score.
Assumptions & free parameters
free parameters (2)
- λ (fairness weight) =
4
- β (VAE KL weight) =
not reported
assumptions (3)
- domain assumption The subsampled RDP bound ε ≤ γ² 2α / σ² (Mironov et al., 2019) is valid for the balanced group-wise sampling scheme when the accountant uses γ_max.
- ad hoc to paper Minimizing negative CKA between protected-group activation matrices is sufficient to drive the generated distribution toward X⊥S.
- domain assumption There exists an unbiased representation of the underlying population that the two-phase training can reach.
Cite this review
Pith. "Pith review of Achieving Hilbert-Schmidt Independence Under R\'enyi Differential Privacy for Fair and Private Data Generation." pith.science (2026). https://pith.science/paper/S5FTL7TK
@misc{pith2026250821815,
author = {Pith},
title = {Pith review of: Achieving Hilbert-Schmidt Independence Under R\'enyi Differential Privacy for Fair and Private Data Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/S5FTL7TK}},
note = {Machine review of arXiv:2508.21815}
}
read the original abstract
As privacy regulations such as the GDPR and HIPAA and responsibility frameworks for artificial intelligence such as the AI Act gain traction, the ethical and responsible use of real-world data faces increasing constraints. Synthetic data generation has emerged as a promising solution to risk-aware data sharing and model development, particularly for tabular datasets that are foundational to sensitive domains such as healthcare. To address both privacy and fairness concerns in this setting, we propose FLIP (Fair Latent Intervention under Privacy guarantees), a transformer-based variational autoencoder augmented with latent diffusion to generate heterogeneous tabular data. Unlike the typical setup in fairness-aware data generation, we assume a task-agnostic setup, not reliant on a fixed, defined downstream task, thus offering broader applicability. To ensure privacy, FLIP employs R\'enyi differential privacy (RDP) constraints during training and addresses fairness in the input space with RDP-compatible balanced sampling that accounts for group-specific noise levels across multiple sampling rates. In the latent space, we promote fairness by aligning neuron activation patterns across protected groups using Centered Kernel Alignment (CKA), a similarity measure extending the Hilbert-Schmidt Independence Criterion (HSIC). This alignment encourages statistical independence between latent representations and the protected feature. Empirical results demonstrate that FLIP effectively provides significant fairness improvements for task-agnostic fairness and across diverse downstream tasks under differential privacy constraints.
Figures
Reference graph
Works this paper leans on
-
[1]
Deep learning with differential privacy
MartinAbadi, AndyChu, IanGoodfellow, H.BrendanMcMahan, IlyaMironov, KunalTalwar, andLiZhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC Conference on Com- puter and Communications Security, CCS ’16, pp. 308–318, New York, NY, USA,
work page 2016
-
[3]
URL http://www.jstor.org/stable/24758720. Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository,
-
[8]
European Parliament and Council of the European Union
URLhttps://data.europa.eu/eli/reg/2016/679/oj. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 an...
work page 2016
-
[9]
Mei Ling Fang, Devendra Singh Dhami, and Kristian Kersting
URLhttps://eur-lex.europa.eu/eli/reg/2024/1689. Mei Ling Fang, Devendra Singh Dhami, and Kristian Kersting. DP-CTGAN: differentially private medical data generation using ctgans. In Martin Michalowski, Syed Sibte Raza Abidi, and Samina Abidi (eds.), Artificial Intelligence in Medicine, pp. 178–188, Cham,
work page 2024
-
[12]
doi: 10.1016/j.mlwa.2024.100608
ISSN 2666-8270. doi: 10.1016/j.mlwa.2024.100608. Tobias Hyrup, Anton D Lautrup, Arthur Zimek, and Peter Schneider-Kamp. A systematic review of privacy- preserving techniques for synthetic tabular health data.Discover Data, 3(1):5, March
-
[13]
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,
work page 2014
-
[14]
URL http://arxiv.org/abs/1312.6114. A. Kiran, P. Rubini, and S. Saravana Kumar. Comprehensive review of privacy, utility, and fairness offered by synthetic data.IEEE Access, 13:15795–15811,
-
[16]
ISSN 0360-0300. doi: 10.1145/3704437. Tai Le Quy, Arjun Roy, Gunnar Friege, and Eirini Ntoutsi. Fair-capacitated clustering. InProceedings of The 14th International Conference on Educational Data Mining (EDM21), pp. 407–414. International Educational Data Mining Society,
Show all 29 references
-
[18]
ISBN 9781450390965
Association for Computing Machinery. ISBN 9781450390965. doi: 10.1145/3485447. 3511958. Chuan Ma, Jun Li, Ming Ding, Bo Liu, Kang Wei, Jian Weng, and H. Vincent Poor. RDP-GAN: a rényi- differential privacy based generative adversarial network.IEEE Transactions on Dependable an...
-
[22]
doi: 10.14778/ 3583140.3583168
ISSN 2150-8097. doi: 10.14778/ 3583140.3583168. Amirarsalan Rajabi and Ozlem Ozmen Garibay. TabFairGAN: fair tabular data generation with generative adversarial networks.Machine Learning and Knowledge Extraction, 4(2):488–501,
-
[23]
doi: 10.3390/make4020022
ISSN 2504-4990. doi: 10.3390/make4020022. Resmi Ramachandranpillai, Md Fahim Sikder, and Fredrik Heintz. Fair latent deep generative models (fldgms) for syntax-agnostic and fair synthetic data generation. InECAI 2023, pp. 1938–1945. IOS Press,
2023 doi
-
[24]
Tab- ularargn: A flexible and efficient auto-regressive framework for generating high-fidelity synthetic data
Paul Tiwald, Ivona Krchova, Andrey Sidorenko, Mariana Vargas-Vieyra, Mario Scriminaci, et al. Tab- ularargn: A flexible and efficient auto-regressive framework for generating high-fidelity synthetic data. arXiv preprint arXiv:2501.12012,
-
[25]
doi: 10.1016/j.ins.2021.12.018
ISSN 0020-0255. doi: 10.1016/j.ins.2021.12.018. Boris van Breugel, Trent Kyono, Jeroen Berrevoets, and Mihaela van der Schaar. DECAF: generating fair synthetic data using causally-aware generative networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman ...
2021 doi
-
[27]
Shuai Wang, Paul Verhagen, Jennifer Zhuge, and Velizar Shulev
URL https://proceedings.mlr.press/v235/vero24a.html. Shuai Wang, Paul Verhagen, Jennifer Zhuge, and Velizar Shulev. Replication study of decaf: Generating fair synthetic data using causally-aware generative networks. InML Reproducibility Challenge 2021 (Fall Edition),
2021
-
[30]
Hengrui Zhang, Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis
URL https://arxiv.org/abs/2109.12298. Hengrui Zhang, Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. InThe Twelfth Internat...
-
[1971]
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner
URL http://www.jstor.org/stable/2528823. Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with a constrained variational framework. In International Co...
-
[2006]
ISBN 978-3-540-32732-5
Springer Berlin Heidelberg. ISBN 978-3-540-32732-5. European Parliament and Council of the European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal dat...
2016
-
[2014]
doi: 10.1561/0400000042
ISSN 1551-305X. doi: 10.1561/0400000042. URL http://dx.doi.org/10.1561/0400000042. 17 Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in pri- vate data analysis. In Shai Halevi and Tal Rabin (eds.),Theory of Cryptography, pp. 265–2...
-
[2015]
ISBN 9781450336642
Association for Computing Machinery. ISBN 9781450336642. doi: 10.1145/2783258.2783311. Georgi Ganev, Bristena Oprisanu, and Emiliano De Cristofaro. Robin hood and matthew effects: Differential privacy has disparate impact on synthetic data. In Kamalika Chaudhuri, Stefanie Jege...
-
[2016]
ISBN 9781450341394
Association for Computing Machinery. ISBN 9781450341394. doi: 10.1145/2976749.2978318. Ahmed Alaa, Boris Van Breugel, Evgeny S. Saveliev, and Mihaela van der Schaar. How faithful is your syn- thetic data? Sample-level metrics for evaluating and auditing generative models. In K...
-
[2017]
Yujin Choi, Jinseong Park, Hoki Kim, Jaewook Lee, and Saerom Park
URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 978fce5bcc4eccc88ad48ce3914124a2-Paper.pdf. Yujin Choi, Jinseong Park, Hoki Kim, Jaewook Lee, and Saerom Park. Fair sampling in diffusion models through switching mechanism. Proceedings of the AAAI Conference on A...
2017
-
[2018]
Jinsung Yoon, James Jordon, and Mihaela van der Schaar
URLhttps://arxiv.org/abs/1802.06739. Jinsung Yoon, James Jordon, and Mihaela van der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In International Conference on Learning Representations,
-
[2019]
Office for Civil Rights (OCR)
URLhttp://arxiv.org/abs/1908.10530. Office for Civil Rights (OCR). Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule,
1908 arXiv
-
[2020]
doi: 10.1109/JBHI.2020.2980262
ISSN 2168-2194. doi: 10.1109/JBHI.2020.2980262. Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential ...
2020
-
[2021]
P Van Der Laan
URLhttps://proceedings.neurips.cc/paper_files/ paper/2021/file/ba9fab001f67381e56e410575874d967-Paper.pdf. P Van Der Laan. The 2001 census in the netherlands. InConference the Census of Population
2021
-
[2022]
FairGAN: Gans-based fairness-aware learning for recommendations with implicit feedback
Jie Li, Yongli Ren, and Ke Deng. FairGAN: Gans-based fairness-aware learning for recommendations with implicit feedback. InProceedings of the ACM Web Conference 2022, WWW ’22, pp. 297–307, New York, NY, USA,
2022
-
[2023]
Cited by: 24; All Open Access, Green Open Access
doi: 10.1109/TDSC.2022.3233580. Cited by: 24; All Open Access, Green Open Access. Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263–275,
2022
-
[2024]
URL https://ojs.aaai.org/index.php/ AAAI/article/view/30202
doi: 10.1609/aaai.v38i20.30202. URL https://ojs.aaai.org/index.php/ AAAI/article/view/30202. Christophe Denis, Romuald Elie, Mohamed Hebiri, and François Hu. Fairness guarantees in multi-class classification with demographic parity.Journal of Machine Learning Research, 25(130):1–46,
-
[2025]
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton
doi: 10.1109/ACCESS.2025.3532128. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.),Proceedings of the 36th International Conference on Machine Lear...
2025
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.