Pith. sign in

REVIEW 4 major objections 5 minor 71 references

CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single imperceptible audio perturbation, optimized over a batch of utterances and refined in the mel-spectrogram domain, blocks zero-shot voice cloning while protected speech stays natural.

desk verdict Plausible idea, but the paper's central 'universal perturbation' claim is unsupported because evaluation is in-sample and per-sample tuned; needs major revision before it can be trusted. read the letter →

arxiv 2505.19119 v1 pith:ZY2HGDM5 submitted 2025-05-25 cs.SD cs.AIeess.AS

Renyuan Li , Zhibo Liang , Haichuan Zhang , Tianyu Shi , Zhiyuan Cheng , Jia Shi , Carl Yang , Mingjie Tang This is my paper
classification cs.SDcs.AIeess.AS
keywords voicecloningdefenseadversarialperturbationzero-shottext-to-speechuniversalmulti-objectiveoptimizationMGDAspeakerprivacymel-spectrogram
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that one imperceptible time-domain perturbation can shield a group of speech recordings from zero-shot voice cloning. The perturbation is built in two stages: a multi-objective optimization over a batch of up to five utterances produces a shared base signal, and a per-sample refinement in the mel-spectrogram domain keeps that signal inaudible while preserving its disruptive effect. Across three zero-shot cloning systems and five datasets, the authors report that protected audio keeps near-original quality (PESQ 3.90, speaker similarity 0.93) while cloned outputs lose the speaker's identity (similarity 0.08), with reported defense success rates of 100% in most cases and above 92% on the other two systems. If true, this would be a proactive, text-agnostic defense that requires no knowledge of the attacker's transcript.

What carries the argument

The carrying object is a universal perturbation vector $\delta$, initialized uniformly in $[-\epsilon, \epsilon]$ with an $L^\infty$ bound of $\epsilon = 0.15$, applied to every utterance in the batch. MGDA supplies the optimization rule: at each iteration it finds the convex combination of per-utterance loss gradients that minimizes gradient conflict, so a single update improves protection for all samples simultaneously. The second stage re-optimizes the perturbation per sample against a multi-scale mel-spectrogram reference (FFT sizes 512, 1024, and 2048), balancing a reference loss that anchors the protected audio to the original in perceptual space against an output loss that keeps cloned outputs far away, with a circular-buffer dynamic weighting to set the trade-off.

What would settle it

Optimize perturbations on a batch of five utterances, then apply the stage-one universal perturbation (and separately the fully refined perturbation) to held-out utterances from speakers excluded from optimization, and measure speaker similarity of the cloned outputs. If the defense success rate on those held-out samples falls far below the reported in-sample level, the universal-generalization claim fails.

Watch

Extended reading notes

Core claim

The central claim is that one shared time-domain perturbation, produced by solving a multi-objective problem with the Multiple Gradient Descent Algorithm (MGDA) and then fine-tuned per sample in the mel-spectrogram domain, degrades zero-shot cloning outputs enough that the synthesized voice no longer matches the target speaker, while the protected audio stays perceptually close to the original. The authors report protected inputs with speaker recognition similarity (SRS) of 0.93 versus cloned outputs at SRS 0.08, near-original PESQ of 3.90 on inputs and collapsed PESQ around 1.07 on clones, and a defense success rate of 100% on YourTTS with strong results on XTTSv2 and IndexTTS. The framework presents itself as the first proactive and content-agnostic defense, needing neither the attacker's text nor paired training data.

Load-bearing premise

The claim of protection robust across speakers and utterances assumes that a perturbation optimized on a batch of five utterances keeps working on utterances and speakers outside that batch without per-sample retuning, and the paper only evaluates on the same utterances used in the optimization.

Editorial extensions

If this is right

  • A base perturbation computed once over a five-utterance batch can be deployed with only lightweight per-sample refinement, avoiding full per-utterance optimization cost.
  • Because the defense never needs the attacker's text prompt, it applies to open-domain cloning scenarios where transcripts are unavailable.
  • Compared with watermarking baselines, which leave cloned speaker identity essentially intact (defense success near 0), this method actively collapses the identity similarity of cloned outputs.
  • The same two-stage pipeline works across three distinct cloning architectures and five datasets, with protected inputs rated close to original audio by human listeners.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The strongest form of the universal claim, cross-architecture transfer (optimize the perturbation on one cloning model and defend against another), is never tested; if it held, the defense would work against unseen attackers, but the paper only demonstrates matched-model defense.
  • Because the second stage refines the perturbation per sample, deployed protection is not literally one shared waveform but a shared base plus per-sample fine-tuning; the universality that matters in practice is whether the base warm-starts protection for utterances never seen in optimization.
  • A held-out evaluation that excludes optimized utterances and speakers would clarify whether the reported defense success rates are a genuine prior or in-sample overfitting.
  • Combining this perturbation with post-hoc watermarking would give a layered defense: the perturbation prevents usable clones, and the watermark traces any clone that still gets made.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CloneShield, a framework that adds a universal time-domain adversarial perturbation to a group of speech utterances to disrupt zero-shot voice cloning, while keeping the protected audio perceptually close to the original. The method has two stages: a multi-objective optimization (MGDA) over a batch of inputs to produce a shared base perturbation, followed by a per-sample mel-spectrogram-domain refinement that balances imperceptibility (multi-scale mel loss) with output divergence. Experiments on YourTTS, XTTSv2, and IndexTTS over several datasets report high protected-input quality (PESQ ≈ 3.9, SRS ≈ 0.93) and strongly degraded cloned outputs (SRS as low as 0.05, DSR up to 100%).

Significance. If the claims were substantiated, CloneShield would be a practically relevant proactive defense against zero-shot voice cloning, with the notable strength of not requiring the attacker's text prompt. The paper ships a concrete algorithmic recipe (Algorithms 1 and 2), a large evaluation matrix across multiple TTS models and datasets, and a human listening study, which are valuable assets. However, the significance is currently limited by two load-bearing gaps: the universal-perturbation claim is never tested on held-out utterances, and the optimization objective in Eq. (3) is stated in a way that appears to pull the cloned output toward a target rather than degrade it. The announced 'defense success rate' is also defined circularly. These issues are fixable in principle, but they must be resolved before the central claims can be accepted.

major comments (4)
  1. [§3.2, Eq. (3) and Algorithm 1] The stated objective is inconsistent with the defense goal. Eq. (3) defines Li(δ) = Loss(f(xi + δ), y_target) and the text says this 'encourages the cloned output to deviate from a pre-defined cloning target.' Minimizing a loss between the cloned output and y_target drives the output toward y_target, not away from it. If y_target is the original speaker's voice or a natural-sounding target, the optimization would make cloning more successful, not less. The paper never defines what y_target is, what loss function is used, or whether the intended operation is maximization or a different target. This is a load-bearing inconsistency that must be corrected and clarified.
  2. [§4.1, Algorithm 2, Table 1] The 'universal perturbation' claim is not supported by the evaluation protocol. The 450 utterances per dataset are organized only into batch configurations, with no described optimization/test split. Algorithm 2 then refines a separate perturbation for every input xi (lines 5–21), and Table 1 reports metrics on those same in-sample inputs. The headline claim of a single perturbation that 'generalizes across multiple utterances' and is 'robust across speakers and utterances' requires a held-out experiment: optimize a shared δ on a training batch, freeze it, and apply it to disjoint utterances and speakers without per-sample refinement. Without such an experiment, the reported DSR/SRS values are consistent with per-sample overfitting rather than with a universal perturbation.
  3. [§4.1, 'Defense Success Rate' definition] The defense success rate is defined circularly: 'We define a defense as successful if the DSR of the synthesized result falls below 0.50.' DSR cannot be defined in terms of itself; the intended quantity is presumably the speaker similarity score (SRS) of the cloned output. As written, the headline metric 'DSR reaches 100%' is uninterpretable. The threshold 0.50 also needs a justification. This affects the central experimental claim and must be corrected.
  4. [§4.1 and §1 contributions] The claim that the method 'does not require access to target texts or cloned output' is not supported by the algorithm or the evaluation. Algorithm 2 requires the TTS model's output M(xi) on the benign input, which is itself a cloned output produced with some chosen text prompt; the evaluation states that 'distinct text prompts' were assigned to each dataset. The defender in a realistic deployment would not know the attacker's prompt, and it is unclear whether the method works when the text prompt used for protection differs from the attacker's actual prompt. This point must either be demonstrated experimentally or removed from the contributions.
minor comments (5)
  1. [Appendix A.2, text around Table 3] The text states that 'STOI values are substantially negative (approx. -16 to -14).' STOI is bounded between 0 and 1 and cannot be negative; these numbers are presumably SDR values. Please correct the metric name.
  2. [§3.3, Algorithm 2 description] The line 'a reference loss mathcalL ∗ ref' contains malformed LaTeX and should read 'reference loss L_ref.'
  3. [§4.1, Metric Selection] The phrase 'We usePyannote.audio model' has a missing space, and the same paragraph uses CER but later text refers to 'word error rate' for CER1; please clarify whether CER1/CER2 are character error rates or word error rates.
  4. [Abstract and Table 1] The abstract claims 'our defense success rate (DSR) is 100%,' but Table 1 reports DSR values of 0.848, 0.793, and 0.860 for IndexTTS. The claim should be qualified to specific models or the table should be consistent with the abstract.
  5. [§5.2] The proposed 'encoder-only' defense strategy is described as an alternative but is not evaluated anywhere in the paper. It would be helpful to state explicitly that this variant is a proposal for future work, not a validated contribution.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported 100% DSR is an in-sample fit: Algorithm 2 fine-tunes a separate perturbation for each evaluated utterance, and the metrics are computed on those same utterances, so the 'universal' claim is not tested on held-out inputs.

  1. fitted input called prediction [§3.3 Algorithm 2; §4.1–4.2, Table 1]
    "Algorithm 2: 'δ ← xadv_i − xi ... for i = 1 to n do ... add δ to δ∗ ... return δ∗'. §4.1: 'From each dataset, we randomly select 450 utterances ... These are organized into batch configurations of 1-per-batch, 3-per-batch, and 5-per-batch, with 50 batches per setting.' Table 1: 'All results are averaged over batches of 5 utterances.'"

    The paper claims a universal perturbation that is robust across speakers and utterances, but the evaluation never separates optimization from test. Algorithm 2 produces a different refined perturbation δ for each input xi (initialized as xadv_i − xi, updated per sample), and §4.1 describes only batch configurations with no held-out split before Table 1 reports DSR/SRS on those same batches. The output-loss term being minimized, Lout = Dist(M(xadv_i), M(xi)), directly targets divergence of the cloned output, so the reported output degradation (SRS ≈ 0.08, DSR ≈ 100%) is a measure of the fitted objective on the fitting inputs, not a prediction of transfer to unseen utterances, speakers, or TTS architectures.

full rationale

Aside from the evaluation-protocol circularity, the paper is self-contained: MGDA is credited to an external reference (Désidéri 2012), no load-bearing self-citation or imported uniqueness theorem appears, and there is no redefinition of a known result. However, the central advertised property—a single shared perturbation protecting a group and generalizing across speakers/utterances—is not actually measured. Stage 2 explicitly fine-tunes the perturbation per sample, and all headline metrics (PESQ, SRS, DSR) are reported on the same utterances used in that fine-tuning, with no disjoint optimization/test split described. Consequently the 'defense success rate' is an in-sample training objective value rather than a validated generalization result. This warrants a mid-range circularity score: the contribution's universal claim is partly forced by construction, while the per-sample attack itself is a legitimate white-box optimization.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central result rests on white-box access to TTS gradients, hand-chosen hyperparameters (epsilon, loss weights, iteration counts), and an in-sample evaluation design. The optimization objective in Eq. (3) is stated in a way that contradicts its intended effect.

free parameters (7)
  • Perturbation bound epsilon = 0.15
    L_infinity norm bound on the time-domain perturbation, chosen by ablation among {0.1, 0.15, 0.3, 0.5} (Table 3). Central to the imperceptibility/defense trade-off.
  • Initial perturbation range = [-0.1, 0.1]
    Uniform initialization for delta in Algorithm 1 (Appendix A.1).
  • Stage-2 loss weights per model = YourTTS ref=7/200, out=1/600; XTTSv2 ref=1/100, out=3/1000; IndexTTS ref=3/25, out=13/100
    Hand-set coefficients for reference loss and output loss in Algorithm 2, differing per TTS model (Appendix A.1).
  • Number of optimization iterations = 60 (stage 1 and 2), 300 for YourTTS saving
    Iteration counts and saving heuristics (e.g., XTTSv2 ref loss threshold 1.65) are manually chosen (Appendix A.1).
  • Learning rate schedule = 0.001, StepLR factor 0.7 every 30 steps
    Chosen optimization hyperparameters for both stages (Appendix A.1).
  • Mel-spectrogram configurations = nfft 2048/1024/512, hop 512/256/128, 80 mel bands, dB floor 80
    Multi-scale mel representations used in the reference loss; fixed by design (Appendix A.1).
  • Predefined cloning target y_target = distinct text prompts per dataset
    The optimization loss in Eq. (3) references a fixed target y_target; the choice of this target influences the perturbation, and the paper does not analyze sensitivity.
assumptions (5)
  • domain assumption White-box access to the target TTS model's gradients
    Algorithms 1 and 2 backpropagate through the TTS model f to update the perturbation (Eq. 3, Algorithm 1 lines 5-9).
  • ad hoc to paper Minimizing Loss(f(x+delta), y_target) degrades cloning
    The paper states this loss makes outputs 'deviate' from y_target, but as written minimization pulls outputs toward y_target; this assumption is internally inconsistent (Eq. 3).
  • domain assumption A small L_infinity-bounded time-domain perturbation can disrupt the TTS decoder without large audible artifacts
    Core premise of all adversarial audio defenses; evaluated only empirically in-sample.
  • domain assumption pyannote speaker embeddings provide a reliable measure of voice identity similarity
    SRS is computed with pyannote/embedding and used for DSR thresholds (Sec 4.1).
  • domain assumption The attacker does not adapt, uses the same TTS model, and applies no preprocessing
    All evaluations clone from the exact protected inputs with the same model; no adaptive or transfer attacks are considered (Sec 4, Conclusion).

how reviews work

0 comments
Cite this review

Pith. "Pith review of CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning." pith.science (2026). https://pith.science/paper/ZY2HGDM5

@misc{pith2026250519119,
  author       = {Pith},
  title        = {Pith review of: CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZY2HGDM5}},
  note         = {Machine review of arXiv:2505.19119}
}
read the original abstract

Recent breakthroughs in text-to-speech (TTS) voice cloning have raised serious privacy concerns, allowing highly accurate vocal identity replication from just a few seconds of reference audio, while retaining the speaker's vocal authenticity. In this paper, we introduce CloneShield, a universal time-domain adversarial perturbation framework specifically designed to defend against zero-shot voice cloning. Our method provides protection that is robust across speakers and utterances, without requiring any prior knowledge of the synthesized text. We formulate perturbation generation as a multi-objective optimization problem, and propose Multi-Gradient Descent Algorithm (MGDA) to ensure the robust protection across diverse utterances. To preserve natural auditory perception for users, we decompose the adversarial perturbation via Mel-spectrogram representations and fine-tune it for each sample. This design ensures imperceptibility while maintaining strong degradation effects on zero-shot cloned outputs. Experiments on three state-of-the-art zero-shot TTS systems, five benchmark datasets and evaluations from 60 human listeners demonstrate that our method preserves near-original audio quality in protected inputs (PESQ = 3.90, SRS = 0.93) while substantially degrading both speaker similarity and speech quality in cloned samples (PESQ = 1.07, SRS = 0.08).

Figures

Figures reproduced from arXiv: 2505.19119 by the authors.

Figure 1
Figure 1. Overview of our CLONESHIELD framework. We inject imperceptible perturbations to disrupt the unauthorized voice replication. The system consists of ❶Universal protective perturbation generation via multi-objective optimization. ❷Perceptual-frequency domain refinement/fine-tune using mel-spectrogram decomposition. ❸Real-world deployment scenarios showcasing how the perturbation thwarts unauthorized voice replication. … view at source ↗
Figure 2
Figure 2. We selected an audio sample for spectrogram visualization. B1, B2, and B3 represent three [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Additional Spectrograms 23 [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 52 canonical work pages

  1. [1]

    A non-intrusive short-time objective intelligibility measure

    Asger Heidemann Andersen, Jan Mark de Haan, Zheng-Hua Tan, and Jesper Jensen. A non-intrusive short-time objective intelligibility measure. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5085--5089. IEEE, 2017

    work page 2017

  2. [2]

    Common voice: A massively-multilingual speech corpus

    Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. Common voice: A massively-multilingual speech corpus. arXiv preprint arXiv:1912.06670, 2019

    arXiv 1912

  3. [3]

    The t05 system for the voicemos challenge 2024: Transfer learning from deep image classifier to naturalness mos prediction of high-quality synthetic speech

    Kaito Baba, Wataru Nakata, Yuki Saito, and Hiroshi Saruwatari. The t05 system for the voicemos challenge 2024: Transfer learning from deep image classifier to naturalness mos prediction of high-quality synthetic speech. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 818--824. IEEE, 2024

    work page 2024

  4. [4]

    Adversarial attacks on automatic speech recognition (asr): A survey

    Amisha Rajnikant Bhanushali, Hyunjun Mun, and Joobeom Yun. Adversarial attacks on automatic speech recognition (asr): A survey. IEEE Access, 2024

    work page 2024

  5. [5]

    Mel-cepstral distortion of german vowels in different information density contexts

    Erika Brandt, Frank Zimmerer, Bistra Andreeva, and Bernd M \"o bius. Mel-cepstral distortion of german vowels in different information density contexts. In Interspeech, pages 2993--2997, 2017

    work page 2017

  6. [6]

    pyannote

    Herv \'e Bredin. pyannote. audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. In 24th INTERSPEECH Conference (INTERSPEECH 2023), pages 1983--1987. ISCA, 2023

    work page 2023

  7. [7]

    Pyannote

    Herv \'e Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, and Marie-Philippe Gill. Pyannote. audio: neural building blocks for speaker diarization. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7124--7128. IEEE, 2020

    work page 2020

  8. [8]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone

    Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren G \"o lge, and Moacir A Ponti. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International Conference on Machine Learning, pages 2709--2720. PMLR, 2022

    work page 2022

Show all 71 references
  1. [9]

    o lge, G \

    Edresson Casanova, Kelly Davis, Eren G \"o lge, G \"o rkem G \"o knar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. Xtts: a massively multilingual zero-shot text-to-speech model. arXiv preprint arXiv:2406.04904, 2024

    2024 arXiv

  2. [10]

    An evaluation of synthetic speech using the pesq measure

    Milos Cernak and Milan Rusko. An evaluation of synthetic speech using the pesq measure. In Proc. European Congress on Acoustics, pages 2725--2728, 2005

    2005

  3. [11]

    A survey on evaluation of large language models

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15 0 (3): 0 1--45, 2024

    2024

  4. [12]

    V2c: Visual voice cloning

    Qi Chen, Mingkui Tan, Yuankai Qi, Jiaqiu Zhou, Yuanqing Li, and Qi Wu. V2c: Visual voice cloning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21242--21251, June 2022

    2022

  5. [13]

    Cloning one’s voice using very limited data in the wild

    Dongyang Dai, Yuanzhe Chen, Li Chen, Ming Tu, Lu Liu, Rui Xia, Qiao Tian, Yuping Wang, and Yuxuan Wang. Cloning one’s voice using very limited data in the wild. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8322--8...

    2022

  6. [14]

    Indextts: An industrial-level controllable and efficient zero-shot text-to-speech system

    Wei Deng, Siyi Zhou, Jingchen Shu, Jinchao Wang, and Lu Wang. Indextts: An industrial-level controllable and efficient zero-shot text-to-speech system. arXiv preprint arXiv:2502.05512, 2025

    2025 arXiv

  7. [15]

    Multiple-gradient descent algorithm (mgda) for multiobjective optimization

    Jean-Antoine D \'e sid \'e ri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350 0 (5-6): 0 313--318, 2012

    2012

  8. [16]

    Parallel tacotron 2: A non-autoregressive neural tts model with differentiable duration modeling

    Isaac Elias, Heiga Zen, Jonathan Shen, Yu Zhang, Ye Jia, RJ Skerry-Ryan, and Yonghui Wu. Parallel tacotron 2: A non-autoregressive neural tts model with differentiable duration modeling. arXiv preprint arXiv:2103.14574, 2021

    2021 arXiv

  9. [17]

    Zero-query adversarial attack on black-box automatic speech recognition systems

    Zheng Fang, Tao Wang, Lingchen Zhao, Shenyi Zhang, Bowen Li, Yunjie Ge, Qi Li, Chao Shen, and Qian Wang. Zero-query adversarial attack on black-box automatic speech recognition systems. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security...

    2024

  10. [18]

    Black-box adversarial defense against voice conversion using latent space perturbation

    Jie Gao, Haiyun Li, Zhisheng Zhang, and Zhiyong Wu. Black-box adversarial defense against voice conversion using latent space perturbation. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE, 2025

    2025

  11. [19]

    Gray and John D

    Augustine H. Gray and John D. Markel. Distance measures for speech processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 24 0 (5): 0 380--391, oct 1976. doi:10.1109/TASSP.1976.1162849

    1976

  12. [20]

    Voice cloning: Training speaker selection with limited multi-speaker corpus

    David Guennec, Lily Wadoux, Aghilas Sini, Nelly Barbot, and Damien Lolive. Voice cloning: Training speaker selection with limited multi-speaker corpus. In 12th ISCA Speech Synthesis Workshop (SSW2023), pages 170--176. ISCA, 2023

    2023

  13. [21]

    A review of multi-objective optimization: Methods and its applications

    Nyoman Gunantara. A review of multi-objective optimization: Methods and its applications. Cogent Engineering, 5 0 (1): 0 1502242, 2018

    2018

  14. [22]

    A comprehensive survey and analysis of generative models in machine learning

    GM Harshvardhan, Mahendra Kumar Gourisaria, Manjusha Pandey, and Siddharth Swarup Rautaray. A comprehensive survey and analysis of generative models in machine learning. Computer Science Review, 38: 0 100285, 2020

    2020

  15. [23]

    Differential privacy techniques for cyber physical systems: A survey

    Muneeb Ul Hassan, Mubashir Husain Rehmani, and Jinjun Chen. Differential privacy techniques for cyber physical systems: A survey. IEEE Communications Surveys & Tutorials, 22 0 (1): 0 746--789, 2019

    2019

  16. [24]

    Membership inference attacks on machine learning: A survey

    Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54 0 (11s): 0 1--37, 2022

    2022

  17. [25]

    Twenty years of digital audio watermarking—a comprehensive review

    Guang Hua, Jiwu Huang, Yun Q Shi, Jonathan Goh, and Vrizlynn LL Thing. Twenty years of digital audio watermarking—a comprehensive review. Signal processing, 128: 0 222--242, 2016

    2016

  18. [26]

    Defending your voice: Adversarial attack on voice conversion

    Chien-yu Huang, Yist Y Lin, Hung-yi Lee, and Lin-shan Lee. Defending your voice: Adversarial attack on voice conversion. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 552--559. IEEE, 2021 a

    2021

  19. [27]

    Unlearnable examples: Making personal data unexploitable

    Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlearnable examples: Making personal data unexploitable. arXiv preprint arXiv:2101.04898, 2021 b

    2021 arXiv

  20. [28]

    The lj speech dataset, 2017

    Keith Ito. The lj speech dataset, 2017. URL https://keithito.com/LJ-Speech-Dataset/

    2017

  21. [29]

    Text-to-speech and speech-to-text converter—voice assistant

    Sagar Janokar, Soham Ratnaparkhi, Manas Rathi, and Alkesh Rathod. Text-to-speech and speech-to-text converter—voice assistant. In Inventive Systems and Control: Proceedings of ICISC 2023, pages 653--664. Springer, 2023

    2023

  22. [30]

    Diff-tts: A denoising diffusion model for text-to-speech

    Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, and Nam Soo Kim. Diff-tts: A denoising diffusion model for text-to-speech. arXiv preprint arXiv:2104.01409, 2021

    2021 arXiv

  23. [31]

    Speaker anonymization using end-to-end zero-shot voice conversion

    Wonjune Kang. Speaker anonymization using end-to-end zero-shot voice conversion. PhD thesis, Massachusetts Institute of Technology, 2022

    2022

  24. [32]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech

    Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference on Machine Learning, pages 5530--5540. PMLR, 2021

    2021

  25. [33]

    Libritts-r: A restored multi-speaker text-to-speech corpus

    Yuma Koizumi, Heiga Zen, Shigeki Karita, Yifan Ding, Kohei Yatabe, Nobuyuki Morioka, Michiel Bacchiani, Yu Zhang, Wei Han, and Ankur Bapna. Libritts-r: A restored multi-speaker text-to-speech corpus. arXiv preprint arXiv:2305.18802, 2023

    2023 arXiv

  26. [34]

    Base tts: Lessons from building a billion-parameter text-to-speech model on 100k hours of data

    Mateusz ajszczak, Guillermo C \'a mbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, \'A lvaro Mart \' n-Cortinas, Ammar Abbas, Adam Michalski, et al. Base tts: Lessons from building a billion-parameter text-to-speech model on 100k hours of data. arXiv pre...

    2024 arXiv

  27. [35]

    Voicebox: Text-guided multilingual universal speech generation at scale

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36, 2024

    2024

  28. [36]

    Voice guard: Protecting voice privacy with strong and imperceptible adversarial perturbation in the time domain

    Jingyang Li, Dengpan Ye, Long Tang, Chuanxi Chen, and Shengshan Hu. Voice guard: Protecting voice privacy with strong and imperceptible adversarial perturbation in the time domain. In IJCAI, pages 4812--4820, 2023

    2023

  29. [37]

    Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models

    Yinghao Aaron Li, Cong Han, Vinay Raghavan, Gavin Mischler, and Nima Mesgarani. Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  30. [38]

    Detecting voice cloning attacks via timbre watermarking

    Chang Liu, Jie Zhang, Tianwei Zhang, Xi Yang, Weiming Zhang, and Nenghai Yu. Detecting voice cloning attacks via timbre watermarking. arXiv preprint arXiv:2312.03410, 2023

    2023 arXiv

  31. [39]

    Audiomarkbench: Benchmarking robustness of audio watermarking

    Hongbin Liu, Moyang Guo, Zhengyuan Jiang, Lun Wang, and Neil Gong. Audiomarkbench: Benchmarking robustness of audio watermarking. Advances in Neural Information Processing Systems, 37: 0 52241--52265, 2024

    2024

  32. [40]

    A deep learning loss function based on the perceptual evaluation of the speech quality

    Juan Manuel Martin-Donas, Angel Manuel Gomez, Jose A Gonzalez, and Antonio M Peinado. A deep learning loss function based on the perceptual evaluation of the speech quality. IEEE Signal processing letters, 25 0 (11): 0 1680--1684, 2018

    2018

  33. [41]

    Matcha-tts: A fast tts architecture with conditional flow matching

    Shivam Mehta, Ruibo Tu, Jonas Beskow, \'E va Sz \'e kely, and Gustav Eje Henter. Matcha-tts: A fast tts architecture with conditional flow matching. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11341--11345. IEEE, 2024

    2024

  34. [42]

    The audio auditor: user-level membership inference in internet of things voice services

    Yuantian Miao, Minhui Xue, Chao Chen, Lei Pan, Jun Zhang, Benjamin Zi Hao Zhao, Dali Kaafar, and Yang Xiang. The audio auditor: user-level membership inference in internet of things voice services. arXiv preprint arXiv:1905.07082, 2019

    1905 arXiv

  35. [43]

    A comprehensive overview of large language models

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. arXiv preprint arXiv:2307.06435, 2023

    2023 arXiv

  36. [44]

    Oppenheim, Ronald W

    Alan V. Oppenheim, Ronald W. Schafer, and John R. Buck. Discrete-Time Signal Processing. Prentice Hall, 2nd edition, 1999

    1999

  37. [45]

    Librispeech: an asr corpus based on public domain audio books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206--5210. IEEE, 2015

    2015

  38. [46]

    Black-box adversarial defense for enhancing robustness in speaker recognition systems with multimodel consensus

    Umang Patel, Avik Hati, and Shruti Bhilare. Black-box adversarial defense for enhancing robustness in speaker recognition systems with multimodel consensus. In Seventeenth International Conference on Machine Vision (ICMV 2024), volume 13517, pages 449--456. SPIE, 2025

    2024

  39. [47]

    Terahertz spectroscopy in biomedical field: a review on signal-to-noise ratio improvement

    Yan Peng, Chenjun Shi, Yiming Zhu, Min Gu, and Songlin Zhuang. Terahertz spectroscopy in biomedical field: a review on signal-to-noise ratio improvement. PhotoniX, 1: 0 1--18, 2020

    2020

  40. [48]

    Powerset multi-class cross entropy loss for neural speaker diarization

    Alexis Plaquet and Herv \'e Bredin. Powerset multi-class cross entropy loss for neural speaker diarization. arXiv preprint arXiv:2310.13025, 2023

    2023 arXiv

  41. [49]

    Grad-tts: A diffusion probabilistic model for text-to-speech

    Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-tts: A diffusion probabilistic model for text-to-speech. In International Conference on Machine Learning, pages 8599--8608. PMLR, 2021

    2021

  42. [50]

    The Kaldi Speech Recognition Toolkit

    Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Luk\' a s Burget, Ond r ej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motl\' c ek, Yanmin Qian, Petr Schwarz, Jan Silovsk\' y , Georg Stemmer, and Karel Vesel\' y . The Kaldi Speech Recognition Toolkit . In IEEE 2011 Workshop o...

    2011

  43. [51]

    Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel P

    Colin Raffel, Brian McFee, Eric J. Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel P. W. Ellis. mir\_eval: A Transparent Implementation of Common MIR Metrics . In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 201...

    2014

  44. [52]

    Fastspeech 2: Fast and high-quality end-to-end text to speech

    Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech 2: Fast and high-quality end-to-end text to speech. arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  45. [53]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

    2022

  46. [54]

    The rise of ai-powered voice assistants: Analyzing their transformative impact on modern customer service paradigms and consumer expectations

    Fatin Aqilah Binti Mohamad Roslan and Norliza Binti Ahmad. The rise of ai-powered voice assistants: Analyzing their transformative impact on modern customer service paradigms and consumer expectations. Quarterly Journal of Emerging Technologies and Innovations, 8 0 (3): 0 33--64, 2023

    2023

  47. [55]

    Survey of imperceptible and robust digital audio watermarking systems

    Euschi Salah, Zermi Narima, Amine Khaldi, and Kafi Med Redouane. Survey of imperceptible and robust digital audio watermarking systems. Multimedia Tools and Applications, pages 1--47, 2024

    2024

  48. [56]

    Proactive detection of voice cloning with localized watermarking

    Robin San Roman, Pierre Fernandez, Hady Elsahar, Alexandre D \'e fossez, Teddy Furon, and Tuan Tran. Proactive detection of voice cloning with localized watermarking. In International Conference on Machine Learning, volume 235, 2024

    2024

  49. [57]

    Telecommunication Standardization Sector

    International Telecommunication Union. Telecommunication Standardization Sector. Methods for subjective determination of transmission quality. International Telecommunication Union, 1996

    1996

  50. [58]

    Silentcipher: Deep audio watermarking

    Mayank Kumar Singh, Naoya Takahashi, Weihsiang Liao, and Yuki Mitsufuji. Silentcipher: Deep audio watermarking. arXiv preprint arXiv, 2406: 0 03822, 2024

    2024

  51. [59]

    A generative model of memory construction and consolidation

    Eleanor Spens and Neil Burgess. A generative model of memory construction and consolidation. Nature human behaviour, 8 0 (3): 0 526--543, 2024

    2024

  52. [60]

    Intelligent english automatic translation system based on tts technology

    Xiyang Sun and Zhuo Wang. Intelligent english automatic translation system based on tts technology. In 2024 3rd International Conference on Artificial Intelligence and Autonomous Robot Systems (AIARS), pages 687--692. IEEE, 2024

    2024

  53. [61]

    Dual threshold log spectral distance voice activity detector based effective statistical speech enhancement

    K Ayyappa Swamy, Samuda Prathima, N Padmaja, and C Sushma. Dual threshold log spectral distance voice activity detector based effective statistical speech enhancement. International Journal of Advanced Science and Technology, 29 0 (03): 0 5640--5653, 2020

    2020

  54. [62]

    An algorithm for intelligibility prediction of time--frequency weighted noisy speech

    Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen. An algorithm for intelligibility prediction of time--frequency weighted noisy speech. IEEE Transactions on audio, speech, and language processing, 19 0 (7): 0 2125--2136, 2011

    2011

  55. [63]

    Naturalspeech: End-to-end text-to-speech synthesis with human-level quality

    Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng, Yuanhao Yi, Lei He, et al. Naturalspeech: End-to-end text-to-speech synthesis with human-level quality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  56. [64]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023 a

    2023 arXiv

  57. [65]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023 b

    2023 arXiv

  58. [66]

    CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit [sound], 2017

    Christophe Veaux, Junichi Yamagishi, and Kirsten MacDonald. CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit [sound], 2017. URL https://doi.org/10.7488/ds/1994

    2017 doi

  59. [67]

    From one stolen utterance: Assessing the risks of voice cloning in the aigc era

    Kun Wang, Meng Chen, Li Lu, Jingwen Feng, Qianniu Chen, Zhongjie Ba, Kui Ren, and Chun Chen. From one stolen utterance: Assessing the risks of voice cloning in the aigc era. In 2025 IEEE Symposium on Security and Privacy (SP), pages 4277--4295. IEEE Computer Society, 2025

    2025

  60. [68]

    Voice-based ai in call center customer service: A natural field experiment

    Lingli Wang, Ni Huang, Yili Hong, Luning Liu, Xunhua Guo, and Guoqing Chen. Voice-based ai in call center customer service: A natural field experiment. Production and Operations Management, 32 0 (4): 0 1002--1018, 2023

    2023

  61. [69]

    Predicting speech intelligibility using a gammachirp envelope distortion index based on the signal-to-distortion ratio

    Katsuhiko Yamamoto, Toshio Irino, Toshie Matsui, Shoko Araki, Keisuke Kinoshita, and Tomohiro Nakatani. Predicting speech intelligibility using a gammachirp envelope distortion index based on the signal-to-distortion ratio. In INTERSPEECH, pages 2949--2953, 2017

    2017

  62. [70]

    Ungeneralizable examples

    Jingwen Ye and Xinchao Wang. Ungeneralizable examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11944--11953, 2024

    2024

  63. [71]

    Mattergen: a generative model for inorganic materials design

    Claudio Zeni, Robert Pinsler, Daniel Z \"u gner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabb \'e , Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint arXiv:2312.03687, 2023

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.