{"id":"5672ae81-6a99-433a-83c7-a63654a52af9","arxiv_id":"2505.08847","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Differential privacy applied to the model or to counterfactual explanations reduces model extraction attack success on two tabular datasets, at the cost of lower accuracy and less useful explanations.","lead":"A workshop paper tests whether adding differential privacy to a machine learning model, to its counterfactual explanations, or to both can slow model extraction attacks. The expected trade-off appears: privacy noise reduces attack success but also lowers model accuracy and explanation quality.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never computes the privacy budget (epsilon/delta) for the DP-SGD and CounterGAN 'noise levels' in Sections 4.1-4.2, so the central claim of a privacy trade-off is unquantified; without an accountant run, the observed MEA reduction may be mere noise rather than differential privacy.","rationale":"The central claim is that applying DP at the model, the explainer, or both mitigates counterfactual-based model extraction while trading off accuracy and explanation quality. For that claim to be about differential privacy, the configurations must actually deliver a formal DP guarantee, and that requires a complete privacy-accounting computation. The issue is not an internal contradiction in the reported plots; the trends are consistent with the claim that additive noise reduces agreement. The problem is that the 'privacy axis' of the trade-off is calibrated only by a noise multiplier, not by an epsilon value, so the magnitude of the privacy guarantee is unknown. The reader's weakest_assumption identifies exactly this, and I agree. The reader's CONDITIONAL verdict is appropriate: this is an incremental empirical study with a real, fixable gap. Running the privacy accountant and reporting epsilon, plus adding error bars and a proper interaction test for the claimed synergy, would strengthen the paper. I would not move to REJECT because the core empirical observation (more noise produces lower agreement and lower utility) is visible in the figures and does not require the missing epsilon to be true; however, without the accounting, the 'privacy' interpretation remains unsupported.","tokens_in":8464,"tokens_out":5264,"duration_ms":55587,"concrete_test":"Re-run the exact configurations from Sections 4.1 and 4.2 (same datasets, architectures, noise multipliers 0.1/0.5/0.9, batch size, epochs, and l2_norm_clip) while invoking TensorFlow Privacy's RDP accountant at delta=1e-5, and report epsilon for both the DP-Model and DP-Explainer training runs. If any configuration yields epsilon above a standard budget (or if the missing hyperparameters prevent the computation), the central privacy claim is unsupported; if the high-noise settings yield epsilon smaller than 1, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Sections 4.1 and 4.2, the DP treatments are introduced through 'noise levels' 0.1, 0.5, and 0.9 passed to TensorFlow Privacy's DPKerasAdamOptimizer, and Section 3 even labels the 'privacy parameter's noise level epsilon' as if the noise level were the privacy budget. No privacy accountant is run for either the model or the CounterGAN setting, so the realized (epsilon, delta) budget for any configuration is never reported. This matters because DP-SGD's privacy guarantee is a function of the noise multiplier together with the number of training steps, the batch size or sampling ratio, and the target delta; a 'noise level' alone does not define a privacy regime. If the actual epsilon for these runs is large (for instance, because the models are trained for many steps with high sampling ratios), then the paper's central 'privacy vs. predictive performance vs. explainability' trade-off is not established: the observed decline in MEA agreement would be evidence that noise degrades the model, not that a formal privacy guarantee is being delivered. The claimed synergistic effect in Section 5.3 is also asserted from visual comparison rather than an interaction test, but the unquantified privacy budget is the more fundamental issue: without it, the paper cannot support its central claim about privacy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the interplay among differential privacy (DP), model extraction attacks (MEA) that leverage counterfactual explanations (CFs), and predictive performance. It compares four scenarios on two tabular datasets (Housing, EEG): no DP, DP applied to the target model via DP-SGD, DP applied to the CF-generating CounterGAN, and DP applied to both simultaneously. The experimental campaign measures model accuracy/precision/recall/F1, MEA agreement, prediction gain, and CF realism under noise levels 0.1, 0.5, and 0.9. The paper reports that higher DP noise reduces MEA success but also lowers model utility and CF quality, and it claims in Section 5.3 that applying DP at both the model and the explainer yields a synergistic improvement in MEA resistance.","tokens_in":8682,"tokens_out":2793,"duration_ms":29675,"significance":"If the central claim is established, the paper provides practical guidance for MLaaS providers on where to inject DP noise to mitigate CF-assisted model extraction. The paper has concrete strengths: it evaluates a realistic knowledge-distillation-based MEA that consumes counterfactuals, it covers two datasets, and it compares four deployment scenarios in a single framework. The qualitative finding that larger noise reduces attack success and utility is visible in the reported plots and is consistent with intuition. However, the paper's quantitative support is currently too weak for the claimed privacy trade-off: no privacy accounting is reported, so the meaning of the 'privacy' axis is unresolved, and the 'synergistic' claim in Section 5.3 is asserted from visual inspection rather than from a statistical interaction test. The contribution is therefore promising but not yet fully supported.","major_comments":[{"comment":"The paper never computes or reports a privacy budget (epsilon, delta) for any of the DP configurations. Section 3 calls the noise level the 'privacy parameter's noise level epsilon', and Section 4.1/4.2 state only that DP-SGD and the CounterGAN generator use 'noise levels of 0.1, 0.5 and 0.9' with TensorFlow Privacy's DPKerasAdamOptimizer. This conflates the noise multiplier with the privacy guarantee. In DP-SGD, the realized epsilon depends on the noise multiplier, the number of training steps, the sampling ratio, and the target delta; a noise level alone does not define a privacy regime. Without running a privacy accountant, the paper cannot support its central claim that DP provides meaningful protection against MEA, because the observed reduction in agreement could simply reflect model degradation rather than a formal privacy guarantee. The authors should report (epsilon, delta) for each reported configuration or explicitly frame the results as an ablation on noise magnitude rather than as a privacy evaluation.","section":"Sections 3, 4.1, and 4.2"},{"comment":"The claimed 'synergistic improvement in resistance to MEA' when both the model and the explainer are protected is load-bearing but is asserted from visual comparison of the agreement plots in Figures 3 and 4 without any statistical test for interaction. The results are also reported as averages of 5 runs without error bars, confidence intervals, or per-run variability, so the reader cannot judge whether the observed differences between DP-Model and DP-Model-Explainer are meaningful. To support the synergy claim, the paper needs either an interaction test (e.g., two-way ANOVA or paired comparisons with the relevant contrast) or at least a clear presentation of variance and effect sizes. This is not a cosmetic issue: the synergy claim is highlighted as the main novel takeaway.","section":"Section 5.3, Discussion"},{"comment":"Section 4.1 states that 'the best-performing model in term of accuracy was chosen' but does not specify whether this selection was performed separately for each DP configuration or globally across all configurations. If the selection is global, comparing DP-Model-0.1, DP-Model-0.5, and DP-Model-0.9 is confounded by the fact that different hyperparameter settings may have been selected for different noise levels. If the selection is per configuration, the same criterion should be described explicitly. This issue affects the quantitative comparisons in Figures 2-5 and should be clarified or corrected, especially because the paper does not report the selected hyperparameter values (learning rate, l2_norm_clip) for any configuration.","section":"Section 4.1, model selection"}],"minor_comments":[{"comment":"There is a typo in 'optmizer' and the phrase 'a the softmax activation function' should be corrected to 'and a softmax activation function'.","section":"Section 4.1"},{"comment":"The text says 'agreement ranges between 50 and 72 when the number of queries increases up to 1000' and later 'agreement starts at 70 with 50 queries', but it is not always clear whether these numbers are percentages or raw agreement counts; please standardize the units and axis labels in Figures 3 and 4.","section":"Section 5.2 and Figures 3-4"},{"comment":"For the Housing dataset, the paper says 'results show a declining trend' but does not report the specific accuracy/precision/recall/F1 values, unlike the EEG case; please include the numbers or a supplementary table for completeness.","section":"Section 5.1"},{"comment":"The realism metric definition uses an autoencoder reconstruction but does not describe how the autoencoder was trained (architecture, training data, whether it is shared between datasets); please add this detail.","section":"Section 4.2"},{"comment":"The threat model description says the attacker generates random data points 'within a range of -3 to 3 for each feature' without specifying whether this range is per dataset, after standardization, or across all features; please clarify, since this range materially affects the query distribution and the MEA results.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is essentially an empirical evaluation built on the authors' own prior work [10], and the novelty lies in the comparative four-scenario study. The main barrier to publication is the missing privacy accounting; this is fixable but requires rerunning the experiments with a privacy accountant or substantially reframing the claims as noise-ablation results. The synergy claim also needs statistical backing. I would support a major revision, not rejection, because the qualitative direction is defensible and the datasets and attack pipeline are concrete."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper shows that adding DP noise to the model, to the counterfactual generator, or to both damps down counterfactual-assisted model extraction, at the expected cost to accuracy and explanation quality. The direction is visible in the plots and consistent with the mechanism of DP, so the central qualitative finding is defensible.\n\nWhat is actually new: it is the first evaluation I know of that applies DP simultaneously to the target model and the explainer in the CF-based MEA setting. The combination is a straightforward composition of DP-SGD and the authors' own private CounterGAN attack pipeline from [10], so the novelty is incremental, but the experimental setup is honest and the trade-offs are reported across two datasets and several query budgets.\n\nWhere it gets soft: the paper never computes a privacy budget. Sections 4.1 and 4.2 treat 'noise levels' 0.1, 0.5, and 0.9 as if they were epsilon, but no privacy accountant is run, so we have no idea what (epsilon, delta) is actually delivered. That is a load-bearing gap: the whole paper is about a privacy-utility trade-off, and 'privacy' is never quantified. The stress-test note is right about this. Also, results are 5-run averages with no error bars or significance tests; the best model is chosen without a clearly defined validation split; the claimed synergistic effect in Section 5.3 is read off the plots rather than tested as an interaction. No code or data is released, which makes the numbers hard to check.\n\nThe citation pattern is fine, but the reliance on the authors' own prior attack and defense makes the evaluation a bit self-referential. That is common in this subfield and not by itself a flaw.\n\nWho this is for: researchers working on XAI security and mitigation strategies for explanation-based extraction. They will get a quick empirical data point, not a deep result. It deserves a serious referee, but the referee should ask for a proper DP accountant, error bars, and a statistical test of the claimed synergy. Without those, the quantitative claims are not established.\n\nMy recommendation: engage with it, but treat the privacy numbers as placeholders. A workshop-level paper with revision, not a journal-ready result.","headline":"A useful but incremental empirical study confirming that DP noise reduces counterfactual-assisted model extraction, undermined by missing privacy accounting and thin statistics.","tokens_in":9304,"tokens_out":1614,"would_cite":false,"duration_ms":17091,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Differential privacy at model and explainer reduces counterfactual-based model extraction, with measurable utility loss.","keywords":["differential privacy","model extraction attacks","counterfactual explanations","knowledge distillation","explainable AI","MLaaS","privacy-utility trade-off","DP-SGD"],"falsifier":"Use the paper's exact DP settings and compute the achieved epsilon/delta for each noise level. If the privacy budget is large (for example, epsilon above 10) at the noise levels studied, the 'privacy protection' column of the trade-off is weak and the interpretation changes. Alternatively, repeat the extraction experiments under a fixed epsilon budget and check whether the reduction in agreement survives.","tokens_in":8217,"feed_emoji":"🛡️","tokens_out":6581,"duration_ms":61799,"temperature":0.7,"pith_summary":"With explanation-equipped ML-as-a-service, counterfactual explanations can be turned against the provider: an attacker queries the model, collects prediction–counterfactual pairs, and trains a substitute via knowledge distillation. This paper tries to establish that injecting differential privacy into the classifier, into the counterfactual explainer, or into both reduces how well that extraction succeeds, with the strongest defense when both are protected. Using two tabular datasets and noise at three levels, it quantifies the price: accuracy, precision, recall, and F1 fall as noise rises, and counterfactual quality degrades in prediction gain and realism. The practical point is that a provider can choose where to spend privacy noise, and the measurements give a first map of that trade-off.","feed_headline":"DP at model plus explainer cuts explanation-based model theft","feed_subtitle":"New measurements pin down the cost: accuracy and counterfactual quality drop as privacy noise rises.","key_machinery":"The machinery is two noise-injection points plus a measurement protocol. On the model side, DP-SGD clips gradients and adds noise during training, at three settings labeled 0.1, 0.5, and 0.9. On the explainer side, the same noise levels are injected into the generator of a GAN that produces counterfactuals. Attack success is measured as agreement between the extracted substitute and the target on a held-out set, with queries ranging from 50 to 1,000; explanation quality is scored by prediction gain and by realism, the reconstruction error of an autoencoder. These pieces work together to turn 'where to add DP noise' into a measurable three-way trade-off.","core_discovery":"Working with a DP-SGD-trained classifier and a differentially private GAN-based counterfactual generator, the paper's central claim is that differential privacy at the model, at the explainer, or at both simultaneously lowers the agreement between a target model and a knowledge-distillation substitute trained on counterfactual explanations. On one tabular dataset, no-DP agreement climbs from 70 to about 80 as queries grow to 1,000, while a DP-explainer noise of 0.9 keeps agreement near 50–72; comparable drops appear on a second dataset. Applying DP to both points produces what the authors call a synergistic improvement in resistance to extraction, beyond either defense alone. The same noise also degrades utility: prediction gain falls from 0.488 to 0.055 on one dataset when explainer noise is 0.9, and realism scores worsen, meaning counterfactuals become less realistic. The paper concludes that privacy protection against CF-assisted extraction is real but inseparable from losses in model quality and explanation quality.","pith_inferences":["The paper never reports the privacy budget (epsilon, delta) that its noise levels imply; a reader cannot yet tell whether the 'privacy' being bought is a strong guarantee or a weak one. An immediate extension is to recompute the experiments under fixed epsilon budgets rather than fixed noise levels.","If the actual epsilon at noise 0.1 is large, the apparent defense may be mostly utility loss rather than provable privacy, which would change the trade-off's interpretation.","The same two-point DP design could be tested against other explanation-based attacks, such as gradient- or saliency-based extraction; the mechanism suggests protection would generalize, but the paper does not test it."],"forward_implications":["Defending the model and the explainer together gives lower extraction agreement than defending either alone, so providers facing CF-assisted extraction should consider both noise points.","Higher DP noise at either point reduces the attacker's fidelity, but only by paying a visible cost in accuracy, precision, recall, and F1.","Counterfactual quality degrades with noise: prediction gain drops and realism scores increase, so privacy protection directly weakens the usefulness of explanations to legitimate users.","Extraction agreement still rises with query count even under DP, so noise slows but does not fully stop a patient attacker."],"supporting_citations":[{"why":"Provides the knowledge-distillation-based model extraction attack using private counterfactual explanations, the core attack and defense pipeline reused throughout the paper.","marker":"[10]"},{"why":"Introduces the CounterGAN architecture that the paper adapts to generate counterfactuals with and without DP noise.","marker":"[20]"},{"why":"Supplies the DP-SGD training method used to apply differential privacy to the target model.","marker":"[4]"},{"why":"Establishes that counterfactual explanations can be exploited to extract models, motivating the threat model.","marker":"[11]"},{"why":"Defines differential privacy, the formal privacy guarantee behind both mitigation strategies.","marker":"[3]"},{"why":"Shows that model explanations leak privacy-sensitive information, justifying the need to protect the explainer as well as the model.","marker":"[7]"}],"fun_headline_variants":["DP on model and explainer blocks explanation-based theft, at a cost","Differential privacy curbs counterfactual extraction, but hurts quality","Synergistic DP on model and explainer blocks extraction, with utility loss","Privacy noise cuts counterfactual-based theft; accuracy and realism drop","Dual DP thwarts CF-based extraction more than either alone, at a price"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results treat the chosen noise levels as privacy parameters without ever reporting the resulting epsilon/delta budget, so the claim that these settings provide meaningful differential privacy is an unverified premise.","fun_headline_variants_meta":{"raw":{"variants":["DP on model and explainer blocks explanation-based theft, at a cost","Differential privacy curbs counterfactual extraction, but hurts quality","Synergistic DP on model and explainer blocks extraction, with utility loss","Privacy noise cuts counterfactual-based theft; accuracy and realism drop","Dual DP thwarts CF-based extraction more than either alone, at a price"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000939,"raw_usage":{"total_tokens":3983,"prompt_tokens":884,"completion_tokens":3099,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":3002}},"tokens_in":500,"tokens_out":3099,"duration_ms":22799,"temperature":1.0,"reasoning_tokens":3002,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:48:17.706277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the paper's exact DP settings and compute the achieved epsilon/delta for each noise level. If the privacy budget is large (for example, epsilon above 10) at the noise levels studied, the 'privacy protection' column of the trade-off is weak and the interpretation changes. Alternatively, repeat the extraction experiments under a fixed epsilon budget and check whether the reduction in agreement survives.","supporting_citations":[{"cited_title":"Knowledge Distillation-Based Model Extraction Attack using GAN-based Private Counterfactual Explanations","cited_arxiv_id":"2404.03348","evidence_quote":"Provides the knowledge-distillation-based model extraction attack using private counterfactual explanations, the core attack and defense pipeline reused throughout the paper."},{"cited_title":"Nemirovsky, N","cited_arxiv_id":null,"evidence_quote":"Introduces the CounterGAN architecture that the paper adapts to generate counterfactuals with and without DP noise."},{"cited_title":"Abadi, A","cited_arxiv_id":null,"evidence_quote":"Supplies the DP-SGD training method used to apply differential privacy to the target model."},{"cited_title":"Aïvodji, A","cited_arxiv_id":null,"evidence_quote":"Establishes that counterfactual explanations can be exploited to extract models, motivating the threat model."},{"cited_title":"Dwork, Differential privacy, in: International colloquium on automata, languages, and programming, Springer, 2006, pp","cited_arxiv_id":null,"evidence_quote":"Defines differential privacy, the formal privacy guarantee behind both mitigation strategies."},{"cited_title":"Shokri, M","cited_arxiv_id":null,"evidence_quote":"Shows that model explanations leak privacy-sensitive information, justifying the need to protect the explainer as well as the model."}],"review_version":1}