{"id":"ddea0475-68c0-4a70-b52d-e19f3180bdff","arxiv_id":"2504.15663","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FADEL replaces softmax with a Dirichlet-based evidential output in anti-spoofing models, improving EER on ASVspoof2019 LA and ASVspoof2021 LA over baseline backbones.","lead":"This paper applies a method called evidential deep learning to fake audio detection, replacing the standard softmax output with a Dirichlet distribution that carries a measure of uncertainty. On two public anti-spoofing benchmarks, the approach lowers equal error rate for two existing models and shows a connection between model uncertainty and error rates across attack types.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline gain over AASIST may rest on a degraded baseline: Table I reports AASIST at 1.47% EER, but the original AASIST paper reports 1.06% on the same protocol; against that number, FADEL's 1.21% would be a regression, not an improvement.","rationale":"I agree with the reader that the paper's comparison is the weak point, but I locate the issue more sharply: not just whether the backbone hyperparameters are optimal under the new loss, but whether the reported AASIST baseline matches the method it is supposed to improve. The gap between 1.47% (Table I) and the 1.06% reported in the AASIST paper is large enough to reverse the conclusion. The reader's weakest assumption (hyperparameters held fixed) is related, since the discrepancy may come from training setup, but the baseline mismatch is the more direct threat to the headline claim. I keep the verdict CONDITIONAL rather than REJECT because the concern is checkable and the authors may be able to show that their AASIST reimplementation is legitimate (e.g., different averaging or data preprocessing); however, the acceptance condition should be the reconciliation with the official AASIST number, not merely adding error bars. Secondary concerns (no error bars, unquantified uncertainty correlation with A07/A16/A17 excluded, ablation differences within noise) reinforce the conditional status but are not the primary reason. If the official AASIST reproduction confirms 1.06%, this paper's main empirical claim should be rejected as stated.","tokens_in":7830,"tokens_out":11617,"duration_ms":112692,"concrete_test":"Reproduce the AASIST baseline with the official AASIST code/checkpoint and its published hyperparameters on ASVspoof2019 LA, using the same GPU and data split as the paper. If the official pipeline gives about 1.06% EER, recompute Table I with that baseline; if AASIST-FADEL cannot beat 1.06%, the central claim fails. Also report the per-seed EER values for both systems so the 0.26-point gap can be assessed for overlap.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that FADEL improves the backbone, specifically AASIST from 1.47% to 1.21% EER on ASVspoof2019 LA. The load-bearing premise is that the baseline column actually represents AASIST as published. Table I reports avg/best of 1.47/1.35 for AASIST, whereas reference [20] reports 1.06% EER on the ASVspoof2019 LA evaluation set. A 0.41-point gap is larger than seed averaging plausibly explains, and the paper states it uses the original architecture and hyperparameters without reconciling the discrepancy. If the official AASIST result is 1.06%, then AASIST-FADEL's 1.18-1.21% is a regression, not an improvement, and the cross-dataset comparison in Table II is likewise against a possibly weak reimplementation. The paper does not provide code, checkpoints, or per-seed errors, so the reader cannot tell whether the comparison is apples-to-apples. This concern is about experimental control, not about the EDL formulation itself; if the baseline mismatch is real, the strongest claim in the abstract does not follow from the tables.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FADEL, a training scheme for fake audio detection that replaces softmax-based weighted cross-entropy with an evidential deep learning objective. The model outputs are treated as evidence defining a Dirichlet distribution over class probabilities, and the loss is the expected weighted cross-entropy under this distribution (Eqs. 6-7). The method is applied to two backbones, Res-TSSDNet and AASIST, and evaluated on ASVspoof2019 LA and cross-dataset ASVspoof2021 LA. The authors report improved EER and min t-DCF over the baselines, reduced overconfidence in predicted probabilities, and a qualitative correlation between average uncertainty and per-algorithm EER.","tokens_in":8096,"tokens_out":3796,"duration_ms":35262,"significance":"If the empirical results hold, FADEL would be a simple, architecture-agnostic plug-in loss that mitigates softmax overconfidence and improves out-of-distribution generalization in a security-sensitive task. The mathematical formulation in Eqs. (6)-(7) is a standard weighted expected cross-entropy under a Dirichlet prior and appears correct. However, the contribution is primarily an application of existing evidential deep learning to audio anti-spoofing rather than a new derivation, and the empirical case is weakened by a likely baseline discrepancy, missing variance reporting, and an unquantified correlation claim. The paper does not provide code or checkpoints, so the reproducibility of the reported gains cannot currently be verified.","major_comments":[{"comment":"The reported AASIST baseline (1.47% EER avg, 1.35% best) is substantially worse than the 1.06% EER reported in the original AASIST paper [20] on the same ASVspoof2019 LA evaluation protocol. Since Section III-B states that FADEL uses the original architecture and hyperparameter configurations of the backbone, the discrepancy is not explained by the described setup. If the published AASIST figure is the correct comparison point, then AASIST-FADEL's 1.21% EER is a regression rather than an improvement, which would invalidate the central claim in the abstract. The authors should reconcile this gap, report per-seed results for the baseline and FADEL, and release code and checkpoints so the comparison can be audited.","section":"Table I / Section III-B"},{"comment":"The manuscript states that all experiments were run three times with different seeds, but only average and best results are reported, with no standard deviations, confidence intervals, or significance tests. Without these, the claim in Section IV-A that FADEL 'significantly improves' performance cannot be assessed, especially when the reported differences are small, as in Table III where best EER values are 1.13%, 1.13%, and 1.18% across activation functions. Please include variance estimates and, where appropriate, paired significance tests.","section":"Section III-B / Tables I-III"},{"comment":"The claimed 'strong correlation' between average uncertainty and EER is not quantified: no correlation coefficient, confidence interval, or statistical test is reported, and the text explicitly excludes A07, A16, and A17 as exceptions. Excluding three of thirteen points without a principled criterion makes the correlation claim unverifiable. Report the correlation (e.g., Pearson and/or Spearman) over all spoofing algorithms A07-A19, and justify any exclusion of data points.","section":"Section IV-C / Fig. 3"}],"minor_comments":[{"comment":"In Eq. (6), B is referred to as the beta function, but for a K-class Dirichlet prior it should be the multivariate beta function; please clarify.","section":"Section II-D / Eq. (6)"},{"comment":"The class weight ratio is stated as 1:9 for spoof to bonafide, but the exact scalar weights wj are not defined; please specify the values used in Eq. (2) and Eq. (7).","section":"Section III-B"},{"comment":"The AASIST baseline row in the cross-dataset table is not explicitly described as being retrained under the same protocol (including class weighting and hyperparameters) as AASIST-FADEL; please state the training setup for all rows.","section":"Table II"},{"comment":"The histograms would be easier to interpret if the caption stated which subset (evaluation or development) the probabilities are computed over and what bin width is used.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The Table I baseline discrepancy is the main risk: if AASIST cannot be reproduced near its published 1.06% EER, the headline result in the abstract does not follow. I would ask the authors for per-seed logs, code, and a direct reconciliation with the published AASIST result during revision. The novelty is incremental, as FADEL is a direct application of existing evidential deep learning, so acceptance should rest on the reliability of the empirical comparison rather than on methodological novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is a legitimate and, as far as I can tell, first application of evidential deep learning to fake audio detection, and the weighted loss in Eq. (6)-(7) is a correct adaptation of Sensoy et al. That part is clean. But the headline result hinges on a baseline that doesn't match the published AASIST number, and until that's reconciled, the reported gains are not trustworthy.\n\nWhat's good: the paper applies FADEL to two backbones and shows consistent EER and t-DCF improvements on ASVspoof2019 LA and the ASVspoof2021 cross-dataset test. The histograms in Fig. 2 provide a nice visual demonstration that FADEL moves probabilities away from 0/1, which is exactly the overconfidence behavior the authors set out to fix. Three seeds with average and best reporting is better than most papers in this area. The writing is straightforward and the derivation is easy to follow.\n\nSoft spots: the main one is experimental control. Table I reports AASIST at 1.47% EER average, while the original AASIST paper reports 1.06% on the same evaluation protocol. That gap is too large for seed variation. If the official number is correct, AASIST-FADEL at 1.21% would be a regression, not a gain. The paper says it uses the original architecture and hyperparameters, but that doesn't explain the discrepancy. Table II inherits the problem because the 2021 LA baseline is trained in the same setup. No code, no checkpoints, and no per-seed numbers are provided, so a reviewer cannot check whether the comparisons are apples-to-apples.\n\nThe uncertainty analysis is also oversold. The paper claims a strong correlation between average uncertainty and EER but excludes three of the thirteen spoofing algorithms (A07, A16, A17) and reports no correlation coefficient. That's not evidence as strong as the abstract implies. The activation-function ablation is inconclusive: exponential wins on EER, softplus wins on t-DCF, ReLU is within noise. That's a minor issue but it should be discussed, not just tabled.\n\nBottom line: the method is worth a serious look, and the EDL formulation is sound. But the experimental evidence as presented doesn't establish the headline. This is a paper for anti-spoofing researchers who care about OOD robustness; they should read it mainly to run the comparison with a properly reproduced AASIST baseline. I'd send it to peer review, with the authors required to reconcile their baseline with the published AASIST number and to release code or per-seed results. Not a desk reject, not an accept.","headline":"A legitimate first application of evidential learning to anti-spoofing, but the headline gain over AASIST may rest on a weak baseline (1.47% vs. the published 1.06%), so the experimental evidence as presented doesn't yet establish the claim.","tokens_in":8632,"tokens_out":3399,"would_cite":false,"duration_ms":29010,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that replacing the softmax classification head with Dirichlet-evidence training makes fake-audio detectors more accurate on unseen spoofing attacks and gives them uncertainty scores that track attack difficulty.","keywords":["fake audio detection","audio anti-spoofing","evidential deep learning","uncertainty estimation","out-of-distribution generalization","Dirichlet distribution","ASVspoof"],"falsifier":"Re-train AASIST and AASIST-FADEL with an epoch and learning-rate sweep at matched budgets (e.g., 100, 200, and 400 epochs); the uncertainty explanation is only required if the gap persists at every matched budget, or if the evidential loss with a tuned schedule still beats the best tuned weighted-cross-entropy run.","tokens_in":7643,"feed_emoji":"🎙️","tokens_out":6766,"duration_ms":53266,"temperature":0.7,"pith_summary":"Fake audio detectors must catch spoofing attacks that were never seen in training, but softmax classifiers tend to be overconfident on such out-of-distribution inputs. This paper proposes FADEL, a training scheme that replaces the softmax classification head with evidential deep learning: the backbone outputs non-negative evidence, which defines a Dirichlet distribution over the bonafide and spoofed class probabilities, and predictions are made from that distribution's mean while uncertainty is read from its shape. Applied to two existing detectors, the scheme lowers equal error rates on ASVspoof2019 LA (AASIST from 1.47% to 1.21% EER; Res-TSSDNet from 3.53% to 2.92%) and on cross-dataset ASVspoof2021 LA (AASIST from 8.08% to 5.60%, beating a 6.10% sharpness-aware baseline). The paper also shows that the average uncertainty across spoofing algorithms tracks the error rate, evidence that the uncertainty estimate is meaningful. If the claim holds, detector calibration improves without changing backbone architecture or inference cost.","feed_headline":"Uncertainty-aware training cuts fake-audio errors 18 percent","feed_subtitle":"Modeling uncertainty with a Dirichlet head also flags when the detector is guessing.","key_machinery":"The load-bearing object is the Dirichlet distribution over class probabilities, used as a subjective-logic prior. For each input, the backbone's last-layer logits pass through a non-negative activation to become evidence $e_k$; the Dirichlet concentration parameters are $\\alpha_k = e_k + 1$. The expected class probability is $\\bar{p}_k = \\alpha_k / S_i$ with $S_i = \\sum_j \\alpha_j$, and uncertainty is $u = K / S_i$, so zero-evidence inputs yield maximal uncertainty. The training loss is the weighted Bayes risk of cross-entropy under this prior, $\\sum_j w_j y_{ij}(\\psi(S_i) - \\psi(\\alpha_{ij}))$, which encourages high evidence for the correct class and low evidence otherwise. This machinery transfers onto any backbone by replacing the softmax head and loss, which is why the paper can report gains on two different detectors.","core_discovery":"The central claim is that the overconfidence of maximum-likelihood softmax training is a measurable cause of fake-audio detection failures in out-of-distribution spoofing scenarios, and that evidential training removes it. In FADEL, the model's final logits are converted into non-negative evidence $e_k$ by an activation like softplus or exponential; the Dirichlet parameters are $\\alpha_k = e_k + 1$, class probabilities are the Dirichlet means $\\bar{p}_k = \\alpha_k / \\sum_j \\alpha_j$, and uncertainty is $u = K / \\sum_j \\alpha_j$. The training objective is the Bayes risk of weighted cross-entropy under the Dirichlet prior, which reduces to $\\sum_j w_j y_{ij}(\\psi(S_i) - \\psi(\\alpha_{ij}))$. With identical architectures and hyperparameters, AASIST-FADEL reaches 1.21% EER on ASVspoof2019 LA (baseline AASIST: 1.47%) and 5.60% EER on cross-dataset ASVspoof2021 LA (AASIST: 8.08%; ASAM: 6.10%). The paper takes the resulting flattening of predicted probabilities away from 0 and 1, together with the correlation between average uncertainty and per-algorithm EER, as evidence that the model has become both more accurate and better aware of when it is unsure.","pith_inferences":["Because FADEL is a drop-in replacement for the loss and output head, the same evidence mechanism could be applied to other anti-spoofing backbones (e.g., RawNet2, transformer-based detectors) and to deepfake audio benchmarks beyond ASVspoof; the paper does not test this.","The uncertainty value could plausibly be used as an operational rejection threshold in ASV pipelines, flagging high-uncertainty utterances for human review rather than binary accept/reject; the paper only demonstrates correlation, not this downstream use.","The reported correlation between uncertainty and EER might also be exploitable for few-shot attack triage during model development, but the paper does not establish that uncertainty is causally related to errors beyond the correlation shown."],"forward_implications":["Across in-domain evaluation, FADEL improves AASIST's EER from 1.47% to 1.21% and Res-TSSDNet's EER from 3.53% to 2.92% on ASVspoof2019 LA.","In cross-dataset evaluation from ASVspoof2019 LA to ASVspoof2021 LA, AASIST-FADEL reaches 5.60% EER, beating AASIST (8.08%) and ASAM (6.10%).","Predicted probabilities for spoof samples spread away from the extreme bins near 0 and 1, while bonafide samples retain higher probabilities, reducing misclassifications on hard examples.","Average uncertainty across spoofing algorithms A07-A19 correlates with per-algorithm EER, meaning the uncertainty value can serve as a per-attack difficulty signal.","Choice of evidence activation matters only slightly: exponential gives the best EER (1.16%), softplus the best min t-DCF (0.0276) in the ablation."],"supporting_citations":[{"why":"Supplies the evidential deep learning framework: evidence, Dirichlet parameters, and the Bayes-risk loss that FADEL adapts.","marker":"[30]"},{"why":"AASIST is the main backbone whose softmax head FADEL replaces and the direct baseline for the headline EER gains.","marker":"[20]"},{"why":"Res-TSSDNet is the second backbone used to show FADEL's improvement is not architecture-specific.","marker":"[16]"},{"why":"ASVspoof2019 LA provides the training/development data and the in-domain OOD evaluation set (A07-A19).","marker":"[10]"},{"why":"ASVspoof2021 LA is the cross-dataset evaluation target for testing generalization to unseen conditions.","marker":"[11]"},{"why":"ASAM is the sharpness-aware baseline that AASIST-FADEL must beat in the cross-dataset table.","marker":"[27]"},{"why":"Subjective logic supplies the uncertainty definition u = K / sum alpha used for the uncertainty analysis.","marker":"[31]"}],"fun_headline_variants":["Evidential deep learning adds a certainty check to fake-audio detectors","Dirichlet uncertainty head helps fake-audio detectors flag unknown spoofs","Fake-audio AI that says 'I'm not sure' reduces spoofing errors","Uncertainty-aware networks better catch unseen voice spoofs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the original backbone training schedule (100 epochs, optimizer settings, and data configuration) is still right after the loss is changed from weighted cross-entropy to evidential loss, so the gains measure the loss change rather than a mismatch in training setup.","fun_headline_variants_meta":{"raw":{"variants":["Evidential deep learning adds a certainty check to fake-audio detectors","Dirichlet uncertainty head helps fake-audio detectors flag unknown spoofs","Fake-audio AI that says 'I'm not sure' reduces spoofing errors","Uncertainty-aware networks better catch unseen voice spoofs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000353,"raw_usage":{"total_tokens":1964,"prompt_tokens":1030,"completion_tokens":934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":854}},"tokens_in":646,"tokens_out":934,"duration_ms":8742,"temperature":1.0,"reasoning_tokens":854,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:19:37.901557+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train AASIST and AASIST-FADEL with an epoch and learning-rate sweep at matched budgets (e.g., 100, 200, and 400 epochs); the uncertainty explanation is only required if the gap persists at every matched budget, or if the evidential loss with a tuned schedule still beats the best tuned weighted-cross-entropy run.","supporting_citations":[],"review_version":1}