REVIEW 3 major objections 6 minor 37 references
Interpolating Speaker Identities in Embedding Space for Data Expansion
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Training on interpolated speaker identities improves speaker verification by up to 5.24%.
desk verdict A clean, well-executed empirical study of identity interpolation for data expansion in speaker verification, held back only by a missing data-volume control and no variance estimates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing identity is spherical linear interpolation on unit-norm speaker embeddings: for source embeddings $e_i, e_j$ with angle $\theta = \arccos(e_i^\top e_j / (|e_i||e_j|))$, the midpoint is $e_{ij} = \frac{\sin((1-\alpha)\theta)}{\sin\theta} e_i + \frac{\sin(\alpha\theta)}{\sin\theta} e_j$, keeping the result on the unit hypersphere where cosine similarity is the metric of comparison. These midpoints feed a frozen TTS model, so each is rendered as audible speech. A layered nearest-neighbor scheme selects pairs so synthetic identities are spread across the embedding space rather than clustered in dense regions, and same-gender pairing preserves the gender attribute in the synthesized identities.
What would settle it
Train matched models on (a) real data plus INSIDE synthetic identities and (b) real data plus the same number of additional real identities or randomly perturbed embeddings voiced by the same TTS; if gain (a) over (b) disappears, the interpolation mechanism is not the cause. A second check is to measure intra-class verification of synthetic identities across different synthesized utterances and their cosine distance to the two parent identities, to test whether each midpoint is a stable, separable identity.
Extended reading notes
Core claim
The central claim is that interpolating same-gender speaker embeddings with spherical linear interpolation (SLERP) and synthesizing the midpoint identities with a TTS model produces training identities that improve downstream models. In the main speaker-verification setting, the Identity-Expanded variant, which adds 40,000 synthetic identities, gives an average relative improvement of 5.24% over the real-data baseline and reduces EER from 0.691% to 0.649% on the VoxCeleb1 original protocol. The same variant improves gender classification by 13.44% on average across in-domain and cross-domain test sets. Nearest-neighbor pairing outperforms random pairing (2.76% vs 1.82% average relative gain), consistent with the paper's claim that filling sparsely populated regions of embedding space is what makes the expansion effective.
Load-bearing premise
The method assumes that the midpoint of two same-gender speaker embeddings, once spoken by the TTS model, is a well-formed new identity whose synthetic speech behaves enough like real speech that any improvement comes from added identity diversity rather than from simply adding more training data.
Editorial extensions
If this is right
- Training on INSIDE-expanded data reduces speaker-verification error on VoxCeleb1 across all three standard protocols, with the largest identity-expanded variant averaging a 5.24% relative improvement over real-data training.
- Nearest-neighbor pair selection yields a 2.76% average relative gain versus 1.82% for random pairing, so pair geometry is a controllable factor in augmentation quality.
- Gender classification improves by 13.44% on average, including cross-domain sets (TIMIT, Samrómur Children, private data), indicating the synthetic identities preserve gender while adding diversity.
- INSIDE composes with standard acoustic augmentation such as noise, reverberation, and speed perturbation, and can be added to existing training pipelines without architectural changes.
Reading between the lines
- The reported gains may reflect added data volume rather than the interpolation mechanism itself, since no control with an equal number of additional real identities or with alternative synthetic pipelines is reported; such a control would settle this.
- Because the paper's Fig. 4 shows synthetic identities have much narrower intra-class cosine-similarity distributions than real speakers, the method likely helps by improving inter-identity separation more than by teaching intra-speaker variability; tasks sensitive to within-speaker change may benefit less.
- A testable extension is to interpolate across gender or language groups to synthesize deliberately underrepresented identities and measure whether downstream performance on those groups improves or degrades.
- The privacy claim is partial: midpoint identities are derived from nearby real speakers, and the pairing procedure reveals which real speakers are close in embedding space; a membership-inference study on the source identities would quantify the residual privacy risk.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes INSIDE, a data expansion method that creates synthetic speaker identities by spherical interpolation (SLERP) between same-gender speaker embeddings extracted with a frozen YourTTS encoder. The interpolated embeddings are fed to the frozen TTS synthesizer to generate speech, and the resulting synthetic corpus is combined with VoxCeleb2 to train speaker verification and gender classification models. Three variants are tested: random-pair synthetic data (Syn), nearest-neighbor pair selection (NN), and a larger identity-expanded set (ID-Exp). The authors report average relative improvements of up to 5.24% on VoxCeleb1 speaker verification and 13.44% on gender classification, and argue that INSIDE is scalable, controllable, and compatible with existing augmentation. The paper includes an honest limitations section noting that synthetic identities have lower intra-class variability than real speakers.
Significance. If the reported gains are attributable to identity interpolation, INSIDE is a practical and scalable data expansion method with no fitted parameters in the interpolation itself, validated on external benchmarks (VoxCeleb1, TIMIT, Samromur Children, and a private dataset) that are not used to construct the synthetic data. The use of a frozen TTS model and the simple, reproducible nature of the pipeline are strengths. However, the central mechanism is not yet isolated from the effect of simply adding more training data, and the per-subset results contain degradations that the averaged metrics mask. The paper's own Fig. 4 and Section VI acknowledge a distributional mismatch between synthetic and real identities, which further weakens the mechanistic claim without additional experiments. The significance of the contribution therefore depends on the missing control experiments.
major comments (3)
- [Section V-A, Table III] The central claim that interpolating speaker identities in embedding space improves downstream performance is not isolable from the effect of adding more training data. Every INSIDE variant approximately doubles the training corpus: Syn and NN add 1,092,009 synthetic samples (5,994 identities) to the 1,092,009-sample VoxCeleb2 baseline, and ID-Exp adds 1,000,000 samples (40,000 identities). No control is run with an equal-sized addition of real utterances, nor with an existing non-interpolated synthetic pipeline such as SynVox2 or SynAug. Without such a control, the reported relative gains (1.82%, 2.76%, 5.24%) could reflect data volume rather than the interpolation mechanism; the fact that gains increase with the number of added identities/samples is consistent with both explanations.
- [Table III and Table V] The headline averaged relative improvement masks per-subset degradations, and no repeated-seed or variance statistics are reported. For example, Syn worsens Vox1-O EER (0.697 vs 0.691) and Vox1-H EER (1.634 vs 1.612); NN worsens Vox1-O EER (0.694 vs 0.691); WavLM-Large ECAPA-TDNN Global ID-Exp worsens Vox1-H EER (1.247 vs 1.237); and in Table V ID-Exp worsens VoxCeleb1 gender classification error (1.42 vs 1.24). Since the conclusion states that INSIDE 'consistently improves performance,' the absence of per-subset consistency or uncertainty estimates makes that claim unsupported as stated.
- [Section VI and Fig. 4] The manuscript's own diagnostic shows that synthetic identities have far narrower intra-class cosine similarity distributions than real speakers, meaning lower intra-speaker variability. This directly challenges the stated mechanism that the method increases identity diversity to improve robustness and generalization. The limitation is acknowledged in Section VI, but the paper does not connect it to the experimental results or test whether correcting this distributional mismatch would change the conclusions. A control experiment that adds intra-class variability to the synthetic data would help determine whether the observed gains are due to identity interpolation or to some other effect of the synthetic corpus.
minor comments (6)
- [Section III-B, Eqs. (1)-(2)] The SLERP formula in Eq. (2) assumes unit-norm vectors. The paper should state explicitly that speaker embeddings are L2-normalized before interpolation; otherwise the denominator in Eq. (2) and the claim about preserving unit norm are not justified for non-unit embeddings.
- [Section III-C, Eq. (6)] The definition of S in Eq. (6) is circular: S appears on both sides of the union. The intended iterative procedure is described in prose, but the equation should be rewritten, e.g., as an iterative union S = ⋃_{n=1}^{nmax} UniquePairs(P_n \ S_{n-1}), to avoid self-reference.
- [Section VI] The first limitation says the relative performance gain is 'limited to 5.24%,' but 5.24% is the largest relative gain reported in the paper. Please clarify what limitation is intended; for the strongest WavLM-Large Global baseline the gain is 3.06%, so stating 'limited to 5.24%' is confusing.
- [Section IV-C] Please specify the sample rate or duration used for the 100-250 word utterance length filter and for the 300-frame and 150-frame input cuts, as these details affect reproducibility.
- [Introduction and Section VII] The paper describes INSIDE as 'privacy-friendly,' but no privacy analysis is provided. Since the synthetic identities are interpolations of real speaker embeddings, the generated speech may still encode identifiable information; consider tempering this claim or adding a privacy evaluation.
- [Table V] The header 'Samrómur Children ↓' contains a stray down arrow that is likely a formatting artifact; please remove it.
Circularity Check
No significant circularity: the method's outputs are tested against external benchmarks and no prediction reduces to an input by construction.
full rationale
INSIDE's derivation chain is self-contained with respect to its central claim. Synthetic identities are constructed by spherical linear interpolation (Eqs. 1-2) between same-gender real embeddings extracted with a frozen TTS speaker encoder; alpha = 0.5 is a fixed design choice, not a fitted parameter. The synthetic utterances are then evaluated by training standard speaker verification and gender classification systems (WavLM-Large front-end, ECAPA-TDNN variants) on VoxCeleb2 plus synthetic data and testing on external benchmarks (VoxCeleb1, TIMIT, Samromur Children, private data) that are not used to construct the synthetic corpus. No test quantity is used as an input to the method, and no 'prediction' is computed from the fit of a parameter. The nearest-neighbor pair selection is described by Eqs. 3-6 in the paper; the citation to the authors' prior work [18] is an inspiration for the traversal strategy, not a load-bearing proof. The same-gender pairing is a transparent design assumption, and the paper explicitly acknowledges the synthetic-distribution mismatch (Fig. 4, Section VI) rather than hiding it. The absence of an equal-volume control is an experimental-design limitation, not a circularity. Therefore no circular step can be exhibited, and the score is 0.
Assumptions & free parameters
free parameters (3)
- Interpolation coefficient alpha =
0.5 (midpoint)
- Target number of synthetic identities T =
5,994 (Syn/NN) and 40,000 (ID-Exp)
- Utterance length filter =
100 to 250 words from LibriSpeech
assumptions (5)
- standard math SLERP on unit-norm embeddings preserves the unit hypersphere and aligns with cosine similarity (Eqs. 1-2).
- domain assumption The YourTTS speaker embedding space is structured enough that midpoints represent plausible new speaker identities.
- domain assumption Grouping interpolation pairs by gender avoids major acoustic boundaries and preserves gender coherence.
- domain assumption Synthetic speech from interpolated embeddings is close enough to natural speech that SV training on it transfers to real test sets.
- domain assumption The WeSpeaker training recipes and VoxCeleb1 evaluation protocols are the right yardsticks.
invented entities (1)
-
Synthetic speaker identities (e.g., 'New Speaker AB')
Cite this review
Pith. "Pith review of Interpolating Speaker Identities in Embedding Space for Data Expansion." pith.science (2026). https://pith.science/paper/WQCPRDSG
@misc{pith2026250819210,
author = {Pith},
title = {Pith review of: Interpolating Speaker Identities in Embedding Space for Data Expansion},
year = {2026},
howpublished = {\url{https://pith.science/paper/WQCPRDSG}},
note = {Machine review of arXiv:2508.19210}
}
read the original abstract
The success of deep learning-based speaker verification systems is largely attributed to access to large-scale and diverse speaker identity data. However, collecting data from more identities is expensive, challenging, and often limited by privacy concerns. To address this limitation, we propose INSIDE (Interpolating Speaker Identities in Embedding Space), a novel data expansion method that synthesizes new speaker identities by interpolating between existing speaker embeddings. Specifically, we select pairs of nearby speaker embeddings from a pretrained speaker embedding space and compute intermediate embeddings using spherical linear interpolation. These interpolated embeddings are then fed to a text-to-speech system to generate corresponding speech waveforms. The resulting data is combined with the original dataset to train downstream models. Experiments show that models trained with INSIDE-expanded data outperform those trained only on real data, achieving 3.06\% to 5.24\% relative improvements. While INSIDE is primarily designed for speaker verification, we also validate its effectiveness on gender classification, where it yields a 13.44\% relative improvement. Moreover, INSIDE is compatible with other augmentation techniques and can serve as a flexible, scalable addition to existing training pipelines.
Figures
Reference graph
Works this paper leans on
-
[1]
An overview of text-independent speaker recognition: From features to supervectors,
T. Kinnunen and H. Li, “An overview of text-independent speaker recognition: From features to supervectors,” Speech Communication, vol. 52, no. 1, pp. 12–40, 2010
work page 2010
-
[2]
Reshape dimensions network for speaker recognition,
I. Yakovlev, R. Makarov, A. Balykin, P. Malov, A. Okhot- nikov, and N. Torgashov, “Reshape dimensions network for speaker recognition,” arXiv preprint arXiv:2407.18223 , 2024
arXiv 2024
-
[3]
Golden Gemini is all you need: Finding the sweet spots for speaker verification,
T. Liu, K. A. Lee, Q. Wang, and H. Li, “Golden Gemini is all you need: Finding the sweet spots for speaker verification,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 2324–2337, 2024
work page 2024
-
[4]
A study of multi- scale feature learning from pre-trained models on speaker verification,
S. Peng, W. Guo, J. Zhang, et al. , “A study of multi- scale feature learning from pre-trained models on speaker verification,” in Proc. ICASSP, 2025, pp. 1–5
work page 2025
-
[5]
J. Peng, L. Mošner, L. Zhang, et al. , “Ca-mhfa: A context- aware multi-head factorized attentive pooling for ssl-based speaker verification,” in Proc. ICASSP, 2025, pp. 1–5
work page 2025
-
[6]
Neural acoustic- phonetic approach for speaker verification with phonetic atten- tion mask,
T. Liu, R. K. Das, K. A. Lee, and H. Li, “Neural acoustic- phonetic approach for speaker verification with phonetic atten- tion mask,” IEEE Signal Processing Letters, vol. 29, pp. 782– 786, 2022
work page 2022
-
[7]
V oxBlink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,
Y . Lin, M. Cheng, F. Zhang, Y . Gao, S. Zhang, and M. Li, “V oxBlink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,” in Proc. Inter- speech, 2024, pp. 4263–4267
work page 2024
-
[8]
S. Zheng, L. Cheng, Y . Chen, H. Wang, and Q. Chen, “3d- speaker: A large-scale multi-device, multi-distance, and multi- dialect corpus for speech representation disentanglement,” arXiv preprint arXiv:2306.15354 , 2023
arXiv 2023
Show all 37 references
-
[9]
Overview of speaker modeling and its applications: From the lens of deep speaker representation learning,
S. Wang, Z. Chen, K. A. Lee, Y . Qian, and H. Li, “Overview of speaker modeling and its applications: From the lens of deep speaker representation learning,” arXiv preprint arXiv:2407.15188, 2024
2024 arXiv
-
[10]
Synvox2: Towards a privacy-friendly voxceleb2 dataset,
X. Miao, X. Wang, E. Cooper, et al. , “Synvox2: Towards a privacy-friendly voxceleb2 dataset,” in Proc. ICASSP , 2024, pp. 11 421–11 425
2024
-
[11]
X-vectors: Robust dnn embeddings for speaker recognition,
D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust dnn embeddings for speaker recognition,” in Proc. ICASSP, 2018, pp. 5329–5333
2018
-
[12]
Audio augmentation for speech recognition,
T. Ko, V . Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition,” in Proc. Interspeech , 2015, pp. 3586–3589
2015
-
[13]
Speaker augmentation and bandwidth extension for deep speaker embedding,
H. Yamamoto, K. A. Lee, K. Okabe, and T. Koshinaka, “Speaker augmentation and bandwidth extension for deep speaker embedding,” in Proc. Interspeech, 2019, pp. 406–410
2019
-
[14]
Data augmentation using deep generative models for embedding based speaker recognition,
S. Wang, Y . Yang, Z. Wu, Y . Qian, and K. Yu, “Data augmentation using deep generative models for embedding based speaker recognition,” IEEE/ACM Transactions on Au- dio, Speech, and Language Processing , vol. 28, pp. 2598– 2609, 2020
2020
-
[15]
Augmenting short enrollment speech via synthesis for target speaker extraction,
Z. Huang, J. Lin, M. Ge, et al., “Augmenting short enrollment speech via synthesis for target speaker extraction,” in Proc. ICASSP, 2025, pp. 1–5
2025
-
[16]
Libri2V ox dataset: Target speaker extraction with diverse speaker conditions and synthetic data,
Y . Liu, X. Liu, X. Miao, and J. Yamagishi, “Libri2V ox dataset: Target speaker extraction with diverse speaker conditions and synthetic data,” arXiv preprint arXiv:2412.12512 , 2024
2024 arXiv
-
[17]
Synthasr: Unlocking synthetic data for speech recognition,
A. Fazel, W. Yang, Y . Liu, et al. , “Synthasr: Unlocking synthetic data for speech recognition,” in Proc. Interspeech, 2021, pp. 896–900
2021
-
[18]
V oice conversion augmentation for speaker recognition on defective datasets,
R. Tao, Z. Shi, Y . Jiang, T. Liu, and H. Li, “V oice conversion augmentation for speaker recognition on defective datasets,” arXiv preprint arXiv:2404.00863 , 2024
2024 arXiv
-
[19]
Synaug: Synthesis-based data augmentation for text-dependent speaker verification,
C. Du, B. Han, S. Wang, Y . Qian, and K. Yu, “Synaug: Synthesis-based data augmentation for text-dependent speaker verification,” in Proc. ICASSP, 2021, pp. 5844–5848
2021
-
[20]
Disentangling voice and content with self-supervision for speaker recogni- tion,
T. Liu, K. A. Lee, Q. Wang, and H. Li, “Disentangling voice and content with self-supervision for speaker recogni- tion,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023, pp. 50 221–50 236
2023
-
[21]
Self-supervised speaker recognition with loss-gated learning,
R. Tao, K. Aik Lee, R. Kumar Das, V . Hautamäki, and H. Li, “Self-supervised speaker recognition with loss-gated learning,” in Proc. ICASSP, 2022, pp. 6142–6146
2022
-
[22]
Deep normaliza- tion for speaker vectors,
Y . Cai, L. Li, A. Abel, X. Zhu, and D. Wang, “Deep normaliza- tion for speaker vectors,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 733–744, 2021
2021
-
[23]
Animating rotation with quaternion curves,
K. Shoemake, “Animating rotation with quaternion curves,” in Proc. SIGGRAPH, 1985, pp. 245–254
1985
-
[24]
Wespeaker: A research and production oriented speaker embedding learning toolkit,
H. Wang, C. Liang, S. Wang, et al., “Wespeaker: A research and production oriented speaker embedding learning toolkit,” in Proc. ICASSP, 2023, pp. 1–5
2023
-
[25]
V oxCeleb2: Deep speaker recognition,
J. S. Chung, A. Nagrani, and A. Zisserman, “V oxCeleb2: Deep speaker recognition,” in Proc. Interspeech , 2018, pp. 1086– 1090
2018
-
[26]
Pushing the frontiers of self-distillation prototypes network with dimension regular- ization and score normalization,
Y . Chen, C. Deng, H. Wang, et al. , “Pushing the frontiers of self-distillation prototypes network with dimension regular- ization and score normalization,” in Proc. Interspeech, 2025, pp. 3688–3692
2025
-
[27]
MFA: TDNN with multi-scale frequency-channel attention for text-independent speaker verification with short utterances,
T. Liu, R. K. Das, K. Aik Lee, and H. Li, “MFA: TDNN with multi-scale frequency-channel attention for text-independent speaker verification with short utterances,” in Proc. ICASSP, 2022, pp. 7517–7521
2022
-
[28]
V oxCeleb: A large-scale speaker identification dataset,
A. Nagrani, J. S. Chung, and A. Zisserman, “V oxCeleb: A large-scale speaker identification dataset,” in Proc. Inter- speech, 2017, pp. 2616–2620
2017
-
[29]
Darpa timit acoustic-phonetic continous speech corpus cd- rom. nist speech disc 1-1.1,
J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, and D. Pallett, “Darpa timit acoustic-phonetic continous speech corpus cd- rom. nist speech disc 1-1.1,” NASA STI/Recon Technical Report N, vol. 93, p. 27 403, 1993
1993
-
[30]
Samrómur children: An Icelandic speech corpus,
C. D. Hernandez Mena, D. E. Mollberg, M. Borský, and J. Guðnason, “Samrómur children: An Icelandic speech corpus,” in Proc. LREC, 2022, pp. 995–1002
2022
-
[31]
YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone,
E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. Gölge, and M. A. Ponti, “YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone,” in Proc. ICML, 2022, pp. 2709–2720
2022
-
[32]
Lib- rispeech: An asr corpus based on public domain audio books,
V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An asr corpus based on public domain audio books,” in Proc. ICASSP, 2015, pp. 5206–5210
2015
-
[33]
WavLM: Large-scale self- supervised pre-training for full stack speech processing,
S. Chen, C. Wang, Z. Chen, et al., “WavLM: Large-scale self- supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022
2022
-
[34]
ECAPA- TDNN: Emphasized channel attention, propagation and ag- gregation in tdnn based speaker verification,
B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA- TDNN: Emphasized channel attention, propagation and ag- gregation in tdnn based speaker verification,” in Proc. Inter- speech, 2020, pp. 3830–3834
2020
-
[35]
The speakin system for voxceleb speaker recognition challange 2021,
M. Zhao, Y . Ma, M. Liu, and M. Xu, “The speakin system for voxceleb speaker recognition challange 2021,” arXiv preprint arXiv:2109.01989, 2021
2021 arXiv
-
[36]
Towards quantifying and reducing language mismatch effects in cross-lingual speech anti-spoofing,
T. Liu, I. Kukanov, Z. Pan, Q. Wang, H. B. Sailor, and K. A. Lee, “Towards quantifying and reducing language mismatch effects in cross-lingual speech anti-spoofing,” in IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 1185–1192
2024
-
[37]
Xi-vector embedding for speaker recognition,
K. A. Lee, Q. Wang, and T. Koshinaka, “Xi-vector embedding for speaker recognition,” IEEE Signal Processing Letters , vol. 28, pp. 1385–1389, 2021. 6
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.