RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
pith:T7DBG47Sreviewed 2026-06-29 05:14 UTCmodel grok-4.3open to challenge →
The pith
RECAST reconstructs black-box models from limited counterfactuals via Wasserstein barycentric prototypes without ongoing queries.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
RECAST builds behavioral surrogate models from counterfactual explanations by embedding them as samples for both classes inside Wasserstein barycentric prototypes. The construction counters decision boundary shifts that arise in low-sample regimes and delivers high surrogate fidelity without requiring online access to the target model during reconstruction. Experiments on real-world datasets confirm that the resulting surrogates remain accurate and stable under limited and noisy conditions while enabling systematic fairness diagnostics.
What carries the argument
Wasserstein barycentric prototypes that treat counterfactual explanations as informative samples for both classes to mitigate decision boundary shifts.
If this is right
- High-fidelity surrogates can be obtained from far fewer samples than standard reconstruction methods require.
- Reconstruction proceeds without repeated online queries to the target model after the initial data collection.
- Group fairness diagnostics become available to auditors even when direct model access is restricted.
- Performance remains stable when input data contains moderate noise levels.
Where Pith is reading between the lines
- The same barycentric construction could be tested with other local explanation types such as feature attributions to broaden reconstruction options.
- The limited-access regime makes the approach relevant for auditing models behind privacy-preserving APIs.
- One could measure whether the method scales to high-dimensional inputs by tracking fidelity decay as feature count grows.
Load-bearing premise
Counterfactual explanations can be treated as representative samples for both classes inside the Wasserstein barycentric construction without introducing systematic bias or overfitting.
What would settle it
A reconstruction trial on a dataset with known severe class imbalance in which the surrogate's accuracy on held-out points falls well below the original model's accuracy.
Figures
read the original abstract
Counterfactual explanations (CFs) help understand machine learning models by identifying minimal input changes that would lead to alternative model outcomes. Recent work demonstrates their utility for reconstructing black-box models, enabling third-party auditing of opaque decision systems for fairness and accountability. Still, CF-based reconstruction may suffer from decision boundary shifts, overfitting, and restrictive assumptions requiring online query access to target platforms. We propose REconstruction via Counterfactual-Aware waSserstein opTimization (RECAST) under limited data and restricted access, a behavioral surrogate model based on Wasserstein barycentric prototypes. Our approach addresses decision boundary shifts by incorporating CFs as informative, though less representative, samples for both classes, maintaining high surrogate fidelity in low-sample regimes without requiring online access during reconstruction. To enhance fairness auditing, our method enables systematic group fairness diagnostics. Experiments on real-world datasets and various setups show that RECAST effectively achieves high fidelity and query efficiency, as well as stable results even when the access is limited and noisy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RECAST, a behavioral surrogate model for black-box model reconstruction under limited data and restricted access. It uses counterfactual explanations (CFs) as informative samples for both classes inside a Wasserstein barycentric prototype construction to address decision boundary shifts, claims high surrogate fidelity without requiring online access during reconstruction, supports systematic group fairness diagnostics, and reports strong experimental results on real-world datasets for fidelity, query efficiency, and stability under limited/noisy access.
Significance. If the central construction is shown to avoid bias from treating boundary-near CFs as class-conditional samples, the work could meaningfully advance practical auditing of opaque models in data-scarce regimes. The focus on Wasserstein geometry for limited-data surrogates and the explicit fairness-auditing application are timely strengths.
major comments (2)
- [Abstract] Abstract: the central claim that 'incorporating CFs as informative, though less representative, samples for both classes' maintains high surrogate fidelity rests on the unshown assumption that the Wasserstein barycentric prototype construction corrects for the fact that CFs are deterministically generated near the decision surface rather than sampled from the class-conditional measure. No reweighting, regularization, or convergence argument is visible that would prevent the barycenter from depending on the CF generator.
- [Abstract] Abstract: under the stated regime of 'limited and noisy access,' the method must demonstrate that surrogate fidelity is not an artifact of the CF distribution. The skeptic concern that CFs are not exchangeable with ordinary samples is load-bearing for the reconstruction claim and requires an explicit correction term or empirical validation protocol (e.g., comparison against a baseline that excludes CFs from the barycenter).
minor comments (1)
- [Abstract] The abstract would benefit from a one-sentence statement of the precise optimization objective or barycentric weighting rule.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on the abstract. We address each major comment below and will incorporate clarifications and additional validation in the revision.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claim that 'incorporating CFs as informative, though less representative, samples for both classes' maintains high surrogate fidelity rests on the unshown assumption that the Wasserstein barycentric prototype construction corrects for the fact that CFs are deterministically generated near the decision surface rather than sampled from the class-conditional measure. No reweighting, regularization, or convergence argument is visible that would prevent the barycenter from depending on the CF generator.
Authors: We agree that the abstract does not make the correction mechanism explicit. The full manuscript (Section 3) constructs the Wasserstein barycenter from both limited samples and CFs via optimal transport, which by design averages across the two classes and thereby reduces sensitivity to CFs being boundary-localized; the geometry itself supplies the robustness without an added reweighting term. To address the referee's point directly, we will revise the abstract to reference this property and add a short clarifying paragraph in the method section on why the barycenter does not collapse to the CF generator. This change will be made in the next version. revision: yes
-
Referee: [Abstract] Abstract: under the stated regime of 'limited and noisy access,' the method must demonstrate that surrogate fidelity is not an artifact of the CF distribution. The skeptic concern that CFs are not exchangeable with ordinary samples is load-bearing for the reconstruction claim and requires an explicit correction term or empirical validation protocol (e.g., comparison against a baseline that excludes CFs from the barycenter).
Authors: The experiments already compare RECAST against limited-data baselines without CFs and report improved fidelity when CFs are included, supporting that the gains are not merely an artifact. However, we acknowledge that a dedicated ablation isolating the barycenter construction with versus without CFs would strengthen the claim. We will add this baseline experiment to the revised manuscript, following the referee's suggestion, to provide the requested empirical validation protocol under limited and noisy access. revision: yes
Circularity Check
No significant circularity detected
full rationale
The provided abstract and description contain no equations, derivations, or explicit self-citations that could be inspected for reduction to inputs by construction. The core claim describes an approach that incorporates CFs into a Wasserstein barycentric construction to address boundary shifts, but presents this as a methodological choice rather than a fitted parameter renamed as prediction or a self-definitional loop. No load-bearing uniqueness theorem, ansatz smuggled via citation, or renaming of known results is visible. The derivation chain cannot be walked because no formal steps are shown; the paper therefore remains self-contained against external benchmarks on the basis of the given text.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Barycenters in the Wasserstein Space , url =
Martial Agueh and Guillaume Carlier , title =. 2011 , url =. doi:10.1137/100805741 , timestamp =
-
[2]
Model extraction from counterfactual explanations , journal =
Ulrich A. Model extraction from counterfactual explanations , journal =
-
[3]
Characterizing the risk of fairwashing , booktitle =
Ulrich A. Characterizing the risk of fairwashing , booktitle =. 2021 , url =
2021
-
[4]
Blanchet and Yang Kang and Karthyek Rajhaa A
Jose H. Blanchet and Yang Kang and Karthyek Rajhaa A. M. , title =. J. Appl. Probab. , volume =. 2019 , url =. doi:10.1017/JPR.2019.49 , timestamp =
-
[5]
Feder Cooper and Katherine Lee and Matthew Jagielski and Milad Nasr and Arthur Conmy and Eric Wallace and David Rolnick and Florian Tram
Nicholas Carlini and Daniel Paleka and Krishnamurthy Dj Dvijotham and Thomas Steinke and Jonathan Hayase and A. Feder Cooper and Katherine Lee and Matthew Jagielski and Milad Nasr and Arthur Conmy and Eric Wallace and David Rolnick and Florian Tram. Stealing part of a production language model , booktitle =. 2024 , url =
2024
-
[6]
Mingliang Chen and Min Wu , title =
-
[7]
Faster Wasserstein Distance Estimation with the Sinkhorn Divergence , booktitle =
L. Faster Wasserstein Distance Estimation with the Sinkhorn Divergence , booktitle =. 2020 , url =
2020
-
[8]
Li Deng , title =. 2012 , url =. doi:10.1109/MSP.2012.2211477 , timestamp =
-
[9]
Model Reconstruction Using Counterfactual Explanations:
Pasan Dissanayake and Sanghamitra Dutta , editor =. Model Reconstruction Using Counterfactual Explanations:. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 , year =
2024
-
[10]
Trained Random Forests Completely Reveal your Dataset , booktitle =
Julien Ferry and Ricardo Fukasawa and Timoth. Trained Random Forests Completely Reveal your Dataset , booktitle =. 2024 , url =
2024
-
[11]
Interpolating between Optimal Transport and
Jean Feydy and Thibault S. Interpolating between Optimal Transport and. The 22nd International Conference on Artificial Intelligence and Statistics,. 2019 , url =
2019
-
[12]
Rui Gao and Anton J. Kleywegt , title =. Math. Oper. Res. , volume =. 2023 , url =. doi:10.1287/MOOR.2022.1275 , timestamp =
-
[13]
Xueluan Gong and Qian Wang and Yanjiao Chen and Wang Yang and Xinchang Jiang , title =. 2020 , url =. doi:10.1109/MCOM.001.2000196 , timestamp =
-
[14]
Awa Khouna and Julien Ferry and Thibaut Vidal , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2502.05325 , eprinttype =. 2502.05325 , timestamp =
-
[15]
Jiacheng Liang and Ren Pang and Changjiang Li and Ting Wang , editor =. Model Extraction Attacks Revisited , booktitle =. 2024 , url =. doi:10.1145/3634737.3657002 , timestamp =
-
[16]
Advances in mathematics , volume=
A convexity principle for interacting gases , author=. Advances in mathematics , volume=. 1997 , publisher=
1997
-
[17]
Mathematical Programming , volume=
Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations , author=. Mathematical Programming , volume=. 2018 , publisher=
2018
-
[18]
FAccT , pages =
Ramaravind Kommiya Mothilal and Amit Sharma and Chenhao Tan , title =. FAccT , pages =
-
[19]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Knockoff nets: Stealing functionality of black-box models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[20]
Martin Pawelczyk and Klaus Broelemann and Gjergji Kasneci , title =
-
[21]
Reza Shokri and Martin Strobel and Yair Zick , title =
-
[22]
Stealing Machine Learning Models via Prediction APIs , booktitle =
Florian Tram. Stealing Machine Learning Models via Prediction APIs , booktitle =
-
[23]
NeurIPS , pages =
Sohini Upadhyay and Shalmali Joshi and Himabindu Lakkaraju , title =. NeurIPS , pages =
-
[24]
Mittelstadt and Chris Russell , title =
Sandra Wachter and Brent D. Mittelstadt and Chris Russell , title =. CoRR , volume =
-
[25]
FAccT , pages =
Yongjie Wang and Hangwei Qian and Chunyan Miao , title =. FAccT , pages =
-
[26]
Lei You and Lele Cao and Mattias Nilsson and Bo Zhao and Lei Lei , title =
-
[27]
Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V
A survey on model extraction attacks and defenses for large language models , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=
-
[28]
Proceedings of the AAAI conference on artificial intelligence , volume=
Activethief: Model extraction using active learning and unannotated public data , author=. Proceedings of the AAAI conference on artificial intelligence , volume=
-
[29]
29th USENIX security symposium (USENIX Security 20) , pages=
High accuracy and high fidelity extraction of neural networks , author=. 29th USENIX security symposium (USENIX Security 20) , pages=
-
[30]
2018 , url =
FICO , title =. 2018 , url =
2018
-
[31]
Proceedings of the 3rd innovations in theoretical computer science conference , pages=
Fairness through awareness , author=. Proceedings of the 3rd innovations in theoretical computer science conference , pages=
-
[32]
proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages=
Certifying and removing disparate impact , author=. proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages=
-
[33]
Advances in neural information processing systems , volume=
Equality of opportunity in supervised learning , author=. Advances in neural information processing systems , volume=
-
[34]
2016 , howpublished =
Julia Angwin and Jeff Larson and Surya Mattu and Lauren Kirchner , title =. 2016 , howpublished =
2016
-
[35]
1996 , howpublished =
Adult Data Set , author =. 1996 , howpublished =
1996
-
[36]
Kelley and Barry, Ronald , title =
Pace, R. Kelley and Barry, Ronald , title =. 1997 , journal =
1997
-
[37]
Proceedings of the 2017 ACM on Asia conference on computer and communications security , pages=
Practical black-box attacks against machine learning , author=. Proceedings of the 2017 ACM on Asia conference on computer and communications security , pages=
2017
-
[38]
Langley , title =
P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =
2000
-
[39]
T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980
1980
-
[40]
M. J. Kearns , title =
-
[41]
Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983
1983
-
[42]
R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000
2000
-
[43]
Suppressed for Anonymity , author=
-
[44]
Newell and P
A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981
1981
-
[45]
A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959
1959
-
[46]
Regulation (EU) 2024/1689 of the European Parliament and of the Council on laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) , year =
2024
-
[47]
SIAM Journal on Scientific Computing , volume=
Iterative Bregman projections for regularized transportation problems , author=. SIAM Journal on Scientific Computing , volume=. 2015 , publisher=
2015
-
[48]
2008 , publisher=
Optimal transport: old and new , author=. 2008 , publisher=
2008
-
[49]
An Invitation to Statistics in Wasserstein Space , pages=
The wasserstein space , author=. An Invitation to Statistics in Wasserstein Space , pages=. 2020 , publisher=
2020
-
[50]
La Matematica , volume=
A duality-based proof of the triangle inequality for the wasserstein distances , author=. La Matematica , volume=. 2024 , publisher=
2024
-
[51]
POT: Python Optimal Transport , journal =
R. POT: Python Optimal Transport , journal =. 2021 , volume =
2021
-
[52]
Data Min
Riccardo Guidotti , title =. Data Min. Knowl. Discov. , volume =
-
[53]
Selbst and Manish Raghavan , title =
Solon Barocas and Andrew D. Selbst and Manish Raghavan , title =. FAT* , pages =
-
[54]
Sahil Verma and Varich Boonsanong and Minh Hoang and Keegan Hines and John Dickerson and Chirag Shah , title =
-
[55]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Interpretable counterfactual explanations guided by prototypes , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[56]
Proceedings of the International Astronomical Union , volume=
Prototype-based Models for the Supervised Learning of Classification Schemes , author=. Proceedings of the International Astronomical Union , volume=. 2016 , publisher=
2016
-
[57]
Advances in Neural Information Processing Systems , volume=
Prototypical Networks for Few-shot Learning , author=. Advances in Neural Information Processing Systems , volume=
-
[58]
Gloria Zen and Elisa Ricci , title =
-
[59]
2016 , url =
I-Cheng Yeh , title =. 2016 , url =
2016
-
[60]
arXiv preprint arXiv:2505.08847 , year=
On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction , author=. arXiv preprint arXiv:2505.08847 , year=
-
[61]
2017 IEEE International symposium on technologies for homeland security (HST) , pages=
How to steal a machine learning classifier with deep learning , author=. 2017 IEEE International symposium on technologies for homeland security (HST) , pages=. 2017 , organization=
2017
-
[62]
, author=
InverseNet: Augmenting Model Extraction Attacks with Training Data Inversion. , author=. IJCAI , pages=
-
[63]
Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , pages=
Dualcf: Efficient model extraction attack from counterfactual explanations , author=. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , pages=
2022
-
[64]
2009 IEEE 12th international conference on computer vision , pages=
Fast and robust earth mover's distances , author=. 2009 IEEE 12th international conference on computer vision , pages=. 2009 , organization=
2009
-
[65]
2009 , publisher =
Optimal Transport: Old and New , author =. 2009 , publisher =
2009
-
[66]
Computational Optimal Transport , author =. Foundations and Trends in Machine Learning , volume =. 2019 , publisher =. doi:10.1561/2200000073 , issn =
-
[67]
Entropy , volume=
On wasserstein two-sample testing and related families of nonparametric tests , author=. Entropy , volume=. 2017 , publisher=
2017
-
[68]
Sinkhorn divergences for unbalanced optimal transport , author=. arXiv preprint arXiv:1910.12958 , year=
-
[69]
Advances in Neural Information Processing Systems , volume=
Differential properties of sinkhorn approximation for learning with wasserstein distance , author=. Advances in Neural Information Processing Systems , volume=
-
[70]
Advances in neural information processing systems , volume=
Stochastic optimization for large-scale optimal transport , author=. Advances in neural information processing systems , volume=
-
[71]
Advances in neural information processing systems , volume=
Sinkhorn distances: Lightspeed computation of optimal transport , author=. Advances in neural information processing systems , volume=
-
[72]
International conference on artificial intelligence and statistics , pages=
Smooth and sparse optimal transport , author=. International conference on artificial intelligence and statistics , pages=. 2018 , organization=
2018
-
[73]
International conference on machine learning , pages=
Fast computation of Wasserstein barycenters , author=. International conference on machine learning , pages=. 2014 , organization=
2014
-
[74]
International Conference on Artificial Intelligence and Statistics , pages=
Learning generative models with sinkhorn divergences , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2018 , organization=
2018
-
[75]
2008 , howpublished =
Fabian Dornhege , title =. 2008 , howpublished =
2008
-
[76]
Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , volume=
A survey on datasets for fairness-aware machine learning , author=. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , volume=. 2022 , publisher=
2022
-
[77]
International Workshop on Advanced Analytics and Learning on Temporal Data , pages=
Ismail-Fawaz, Ali and Ismail Fawaz, Hassan and Petitjean, Fran. International Workshop on Advanced Analytics and Learning on Temporal Data , pages=. 2023 , organization=
2023
-
[78]
arXiv preprint arXiv:1910.12366 , year=
Thieves on sesame street! model extraction of bert-based apis , author=. arXiv preprint arXiv:1910.12366 , year=
-
[79]
Operations research & management science in the age of analytics , pages=
Wasserstein distributionally robust optimization: Theory and applications in machine learning , author=. Operations research & management science in the age of analytics , pages=. 2019 , publisher=
2019
-
[80]
2018 IEEE international symposium on technologies for homeland security (HST) , pages=
Active deep learning attacks under strict rate limitations for online API calls , author=. 2018 IEEE international symposium on technologies for homeland security (HST) , pages=. 2018 , organization=
2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.