Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

M3SD: Multi-modal, Multi-scenario and Multi-language Speaker Diarization Dataset

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an automated audio-video pipeline can produce accurate speaker-turn labels at scale, yielding a 770+ hour multi-scenario, multi-language dataset that fills a gap left by existing speaker diarization corpora.

desk verdict The dataset could be genuinely useful, but the paper never checks the pseudo-labels, so its main claim is unsupported. read the letter →

arxiv 2506.14427 v2 pith:YSC2RN4D submitted 2025-06-17 eess.AS cs.MM

Shilong Wu This is my paper · ORCID
classification eess.AScs.MM
keywords speakerdiarizationdatasetconstructionpseudo-labelsaudio-visualfusionmulti-modalmulti-scenariomulti-languagesemi-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to solve two problems that limit speaker diarization: the shortage of large training datasets and the poor generalization of models across scenarios and languages. It proposes a fully automated pipeline that crawls real online videos, cleans them with quality and synchronization filters, and generates speaker-turn labels by combining audio-only and audio-visual diarization models. The resulting M3SD dataset contains 1,372 records and over 770 hours of speech covering meetings, interviews, debates, speeches, movies, and daily conversations in multiple languages. If the pseudo-labels are accurate, the field gains a large, diverse, open training resource that could reduce the need for expensive manual annotation.

What carries the argument

The load-bearing mechanism is a multi-stage, audio-video-guided dataset construction pipeline. It starts with multilingual keyword-based video collection, then cleans data through shot detection, speech quality scoring, video quality scoring, and audio-visual synchronization detection with hand-set thresholds. Preprocessing extracts face tracks and lip regions of interest, which feed two parallel label generators: an end-to-end audio-visual speaker diarization model and an audio-only multi-speaker diarization network that combines clustering with overlap detection. Their outputs are merged by a voting-fusion algorithm into final pseudo-labels, with an optional iterative fine-tuning loop intended to improve both model performance and label quality over time.

What would settle it

Select a random subset of the 1,372 records and have independent human annotators mark speaker turns, including overlaps and off-screen voices, then compute the agreement between the automated M3SD labels and the human labels per scenario; the claim of accurate pseudo-labels is settled only if agreement is high in difficult scenes, not just in clean interviews.

Watch

Extended reading notes

Core claim

The central claim is that combining audio and video information through an automated construction pipeline generates more accurate pseudo-labels for massive speaker diarization data than audio-only labeling alone, and at a scale that manual annotation cannot reach. The paper reports that this pipeline, applied to videos from public internet platforms, produced M3SD, a dataset of 1,372 records and 770+ hours, covering multi-modal, multi-scenario, and multi-language content. The paper further claims that this dataset can fill the gap of insufficient data scenarios in existing datasets and that the accompanying open-sourced construction code supports building even larger datasets without manual annotation.

Load-bearing premise

The entire dataset's value rests on the automatically generated speaker-turn labels being accurate enough to train on, but the paper provides no human-annotated reference set to verify this.

Editorial extensions

If this is right

  • If the pseudo-labels are accurate, training on M3SD should improve diarization generalization across diverse real-world scenes rather than only meeting rooms.
  • The fusion of audio-only and audio-visual outputs should make labels more robust to segments where one modality fails, such as noisy audio or occluded faces.
  • The open-sourced construction code could let other groups build similarly large datasets for new languages and scenarios at low cost.
  • The iterative retraining loop implies a path to continuously expanding training data without manual annotation, if label quality is maintained.
  • Including video metadata makes the dataset usable for audio-visual diarization research, not just audio-only model training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the paper does not demonstrate that its pseudo-labels are accurate enough to train on; a natural test is to fine-tune a standard end-to-end diarization model on M3SD and measure performance on held-out public benchmarks.
  • Editorial extension: the iterative pseudo-label retraining loop could amplify model biases if the two source models share the same failure modes, so a human-annotated holdout set would be needed to confirm that the loop improves rather than entrenches errors.
  • Editorial extension: the cleaning thresholds (speech score at 3, video score at 60, sync offset at 5, confidence at 1) appear to be chosen without reported sensitivity analysis, so the pipeline's behavior at neighboring threshold values remains unknown.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces M3SD, a claimed multi-modal, multi-scenario, multi-language speaker diarization dataset built by an automated pipeline that crawls web videos, cleans them with DNSMOS/MD-VQA/SyncNet thresholds, and generates pseudo-labels using a pre-trained audio-visual diarization model and a pre-trained audio-only model fused with DOVER-LAP. The manuscript reports 1,372 records and 770+ hours of data and states that the dataset and code are open-sourced. The paper contains no experiments, no quantitative evaluation of pseudo-label quality, no downstream fine-tuning results, and no dataset statistics beyond total hours and number of records.

Significance. If properly validated, M3SD could address a real gap in speaker-diarization resources by providing a large, diverse, multi-modal training corpus. However, the paper provides no evidence that the pseudo-labels are accurate or that the dataset improves generalization. The central claims rest entirely on the unverified premise that the automated pipeline produces usable labels; as submitted, the contribution is a pipeline description and a link to a dataset rather than a demonstrated research resource.

major comments (4)
  1. [Abstract and Section III.D] The central claim that the audio-video guided pipeline 'generates more accurate pseudo-labels for massive data' is not tested anywhere in the manuscript. There is no human-annotated held-out subset, no DER or Jaccard error computation against any reference, no per-scenario or per-language label-quality analysis, and no comparison with alternative pseudo-labeling methods. Without such evidence, the dataset's value as a training resource is unsupported; adding a label-quality evaluation on a manually annotated subset, or a fine-tuning experiment evaluated on public benchmarks (e.g., AMI, DIHARD), is essential to substantiate the paper's main assertion.
  2. [Section III.B] The cleaning thresholds (DNSMOS score of 3, MD-VQA score of 60, SyncNet offset less than or equal to 5, confidence greater than or equal to 1) are introduced as fixed choices without any calibration, sensitivity analysis, or relation to downstream pseudo-label quality. Since these thresholds determine which data survive to be labeled, the authors should report score distributions and justify the thresholds empirically, for example by showing how varying them affects pseudo-label accuracy or downstream model performance.
  3. [Section III.D and Section I] The paper promises a 'fine-tuning method for scene-related speaker diarization models' in the Introduction and describes in Section III.D an 'iterative training' loop for improving pseudo-label quality, but neither is implemented, described in detail, or measured. The reader is left with a pipeline description for dataset construction only; the claimed fine-tuning contribution is absent from the paper.
  4. [Section III.A and Section III.D] The dataset statistics are insufficient to support the multi-scenario and multi-language claims. The paper reports only total records (1,372) and total hours (770+), with no breakdown by scenario (meeting, debate, movie, etc.), by language, by number of speakers, by recording duration, or by overlap ratio. Without these statistics, the claimed diversity and coverage cannot be assessed, and the paper does not provide the descriptive analysis expected of a dataset paper.
minor comments (4)
  1. [Section III.B] In the Audio/Video Extraction paragraph, 'sampling rate of 44k' should presumably read '44.1 kHz' to be technically accurate.
  2. [References] Reference [8] ('Speaker diarization with lstm') is duplicated from reference [3]; please renumber or remove the duplicate.
  3. [Section II] The dataset name 'Alimeeting' should be written in its standard form 'AliMeeting' for consistency with the cited M2MeT challenge.
  4. [General] The manuscript cites Figures 1-4 and 2-3 but the figures are not visible in the provided text; please ensure all figures are included and clearly readable in the final submission.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset construction pipeline relies on external pretrained models and open benchmarks; the unvalidated pseudo-label accuracy is an empirical gap, not a circular reduction.

full rationale

The paper's construction method is an automated pipeline: video acquisition, cleaning (DNSMOS, MD-VQA, SyncNet), preprocessing (RetinaFace, DeepSORT, MediaPipe), and pseudo-label generation by fusing a pretrained audio-visual diarization model [26] and the 3D-Speaker audio-only model [51] with DOVER-LAP [52]. The labeling models are trained on external datasets (AMI, MISP, MSDWild, AVA-AVD) and are not fitted to any subset of the released M3SD data in a way that would make the dataset's labels equivalent to its own inputs. The claim that pseudo-labels are 'more accurate' is asserted rather than measured; no DER or human-annotated subset is provided. That is an empirical validation gap, not a self-referential derivation: there is no equation in which the predicted quantity is defined in terms of the fitted quantity. The 'iterative training' paragraph describes a self-training loop, but it is not implemented or used as evidence for any result in the paper, so it cannot be load-bearing circularity. The only self-citation is reference [29], used as related work to distinguish this approach from prior semi-supervised diarization; it is not invoked to justify the central claim. No uniqueness theorem, ansatz, or renaming of a known result is present. Therefore, despite the unverified pseudo-label quality, the paper does not reduce, by construction, to its own inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central contribution is a dataset plus a pipeline; the ledger shows the pipeline rests on several unvalidated hand-set thresholds and a strong generalization assumption about pre-trained models.

free parameters (4)
  • shot detection threshold = 30
    Hand-set threshold in PySceneDetect to decide scene changes; no sensitivity analysis. Affects which clips are retained and hence label quality.
  • DNSMOS quality threshold = 3
    Minimum acceptable speech quality score (0-5) for data retention; chosen without justification or ablation.
  • MD-VQA quality threshold = 60
    Minimum acceptable video quality score for data retention; no evidence that 60 is optimal.
  • SyncNet offset and confidence thresholds = offset <= 5, confidence >= 1
    Criteria for considering a video synchronized; arbitrary choices that drive data cleaning.
assumptions (3)
  • domain assumption Pre-trained audio-visual diarization model generalizes to web videos
    Section III.D relies on the model trained on AMI, MISP, MSDWild, AVA-AVD to produce reliable predictions on unseen YouTube/Bilibili content.
  • domain assumption Audio and lip movement are strongly correlated
    Section I and III.D assume lip ROI is a valid visual cue for speaker activity; this is the basis for audio-visual diarization.
  • ad hoc to paper Automated cleaning with selected thresholds removes only poor-quality data
    No ablation or manual inspection validates that thresholds 30, 3, 60, 5, 1 preserve data useful for diarization training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M3SD: Multi-modal, Multi-scenario and Multi-language Speaker Diarization Dataset." pith.science (2026). https://pith.science/paper/YSC2RN4D

@misc{pith2026250614427,
  author       = {Pith},
  title        = {Pith review of: M3SD: Multi-modal, Multi-scenario and Multi-language Speaker Diarization Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YSC2RN4D}},
  note         = {Machine review of arXiv:2506.14427}
}
read the original abstract

In the field of speaker diarization, the development of technology is constrained by two problems: insufficient data resources and poor generalization ability of deep learning models. To address these two problems, firstly, we propose an automated method for constructing speaker diarization datasets, which generates more accurate pseudo-labels for massive data through the combination of audio and video. Relying on this method, we have released Multi-modal, Multi-scenario and Multi-language Speaker Diarization (M3SD) datasets. This dataset is derived from real network videos and is highly diverse. Our dataset and code have been open-sourced at https://huggingface.co/spaces/OldDragon/m3sd.

Figures

Figures reproduced from arXiv: 2506.14427 by the authors.

Figure 1
Figure 1. Flowchart of dataset construction process. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example of content covered by the dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Data cleaning and data preprocessing flowchart. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Pseudo-label generation based on the pre-trained speaker diarization model. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Pretrained audio-visual speech encoders adapted with LoRA improve multimodal voice activity projection for turn-taking prediction across multiple languages and a robot mediation corpus.

Reference graph

Works this paper leans on

52 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [1]

    A review of speaker diarization: Recent advances with deep learning,

    T. J. Park, N. Kanda, D. Dimitriadis, K. J. Han, S. Watanabe, and S. Narayanan, “A review of speaker diarization: Recent advances with deep learning,” Computer Speech & Language , vol. 72, p. 101317, 2022

    work page 2022

  2. [2]

    Speaker diarization: A review of recent research,

    X. Anguera, S. Bozonnet, N. Evans, C. Fredouille, G. Friedland, and O. Vinyals, “Speaker diarization: A review of recent research,” IEEE Transactions on audio, speech, and language processing , vol. 20, no. 2, pp. 356–370, 2012

    work page 2012

  3. [3]

    Speaker diarization with lstm,

    Q. Wang, C. Downey, L. Wan, P. A. Mansfield, and I. L. Moreno, “Speaker diarization with lstm,” in 2018 IEEE International conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2018, pp. 5239–5243. 7

    work page 2018

  4. [4]

    Front- end factor analysis for speaker verification,

    N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front- end factor analysis for speaker verification,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 19, no. 4, pp. 788–798, 2010

    2010

  5. [5]

    X- vectors: Robust dnn embeddings for speaker recognition,

    D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X- vectors: Robust dnn embeddings for speaker recognition,” in 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2018, pp. 5329–5333

    work page 2018

  6. [6]

    Developing on-line speaker diarization system

    D. Dimitriadis and P. Fousek, “Developing on-line speaker diarization system.” in Interspeech, 2017, pp. 2739–2743

    work page 2017

  7. [7]

    A study of the cosine distance-based mean shift for telephone speech diarization,

    M. Senoussaoui, P. Kenny, T. Stafylakis, and P. Dumouchel, “A study of the cosine distance-based mean shift for telephone speech diarization,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 22, no. 1, pp. 217–227, 2013

    work page 2013

  8. [8]

    Speaker diarization with lstm,

    Q. Wang, C. Downey, L. Wan, P. A. Mansfield, and I. L. Moreno, “Speaker diarization with lstm,” in 2018 IEEE International conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2018, pp. 5239–5243

    work page 2018

Show all 52 references
  1. [9]

    A robust stopping criterion for agglomerative hierarchical clustering in a speaker diarization system

    K. J. Han and S. S. Narayanan, “A robust stopping criterion for agglomerative hierarchical clustering in a speaker diarization system.” in Interspeech, 2007, pp. 1853–1856

    2007

  2. [10]

    Characterizing performance of speaker diarization systems on far-field speech using standard methods,

    M. Maciejewski, D. Snyder, V . Manohar, N. Dehak, and S. Khudanpur, “Characterizing performance of speaker diarization systems on far-field speech using standard methods,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018, p...

    2018

  3. [11]

    Discriminative neural clustering for speaker diarisation,

    Q. Li, F. L. Kreyssig, C. Zhang, and P. C. Woodland, “Discriminative neural clustering for speaker diarisation,” in 2021 IEEE Spoken Lan- guage Technology Workshop (SLT) . IEEE, 2021, pp. 574–581

    2021

  4. [12]

    Bayesian hmm clustering of x-vector sequences (vbx) in speaker diarization: theory, implemen- tation and analysis on standard tasks,

    F. Landini, J. Profant, M. Diez, and L. Burget, “Bayesian hmm clustering of x-vector sequences (vbx) in speaker diarization: theory, implemen- tation and analysis on standard tasks,” Computer Speech & Language , vol. 71, p. 101254, 2022

    2022

  5. [13]

    End-to-End Neural Speaker Diarization with Permutation-Free Objec- tives,

    Y . Fujita, N. Kanda, S. Horiguchi, K. Nagamatsu, and S. Watanabe, “End-to-End Neural Speaker Diarization with Permutation-Free Objec- tives,” in Proc. Interspeech, 2019, pp. 4300–4304

    2019

  6. [14]

    End-to-end neural speaker diarization with self-attention,

    Y . Fujita, N. Kanda, S. Horiguchi, Y . Xue, K. Nagamatsu, and S. Watan- abe, “End-to-end neural speaker diarization with self-attention,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2019, pp. 296–303

    2019

  7. [15]

    Auxiliary loss of transformer with residual connection for end-to-end speaker diarization,

    Y . Yu, D. Park, and H. Kook Kim, “Auxiliary loss of transformer with residual connection for end-to-end speaker diarization,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 8377–8381

    2022

  8. [16]

    Incorporating end-to-end framework into target- speaker voice activity detection,

    W. Wang and M. Li, “Incorporating end-to-end framework into target- speaker voice activity detection,” in ICASSP 2022-2022 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8362–8366

    2022

  9. [17]

    Target-Speaker V oice Activity Detection: A Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario,

    I. Medennikov, M. Korenevsky, T. Prisyach, Y . Khokhlov, M. Ko- renevskaya, I. Sorokin, T. Timofeeva, A. Mitrofanov, A. Andrusenko, I. Podluzhny, A. Laptev, and A. Romanenko, “Target-Speaker V oice Activity Detection: A Novel Approach for Multi-Speaker Diarization in a Dinner ...

    2020

  10. [18]

    Ansd-ma-mse: Adaptive neural speaker diarization using memory-aware multi-speaker embed- ding,

    M.-K. He, J. Du, Q.-F. Liu, and C.-H. Lee, “Ansd-ma-mse: Adaptive neural speaker diarization using memory-aware multi-speaker embed- ding,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, 2023

    2023

  11. [19]

    The chime- 7 dasr challenge: Distant meeting transcription with multiple devices in diverse scenarios,

    S. Cornell, M. Wiesner, S. Watanabe, D. Raj, X. Chang, P. Garcia, Y . Masuyama, Z.-Q. Wang, S. Squartini, and S. Khudanpur, “The chime- 7 dasr challenge: Distant meeting transcription with multiple devices in diverse scenarios,” arXiv preprint arXiv:2306.13734 , 2023

    2023 arXiv

  12. [20]

    The ustc-nercslip systems for chime-7 challenge,

    R. Wan, M. He, J. Du, H. Zhou, S. Niu, H. Chen, Y . Yue, G. Yang, S. Wu, L. Sun et al., “The ustc-nercslip systems for chime-7 challenge,” in Proc. CHiME 2023 , 2023, pp. 13–18

    2023

  13. [21]

    Audio-visual speaker diarization based on spatiotemporal bayesian fusion,

    I. D. Gebru, S. Ba, X. Li, and R. Horaud, “Audio-visual speaker diarization based on spatiotemporal bayesian fusion,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 5, pp. 1086– 1099, 2017

    2017

  14. [22]

    Quantitative association of vocal-tract and facial behavior,

    H. Yehia, P. Rubin, and E. Vatikiotis-Bateson, “Quantitative association of vocal-tract and facial behavior,” Speech Communication, vol. 26, no. 1-2, pp. 23–43, 1998

    1998

  15. [23]

    Multimodal speaker diarization,

    A. Noulas, G. Englebienne, and B. J. Krose, “Multimodal speaker diarization,” IEEE Transactions on Pattern Analysis and Machine In- telligence, vol. 34, no. 1, pp. 79–93, 2011

    2011

  16. [24]

    Who said that?: Audio-visual speaker diarisation of real-world meetings,

    J. S. Chung, B. J. Lee, and I. Han, “Who said that?: Audio-visual speaker diarisation of real-world meetings,” inProceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 2019, 2019, pp. 371–375

    2019

  17. [25]

    Spot the conversation: Speaker diarisation in the wild,

    J. S. Chung, J. Huh, A. Nagrani, T. Afouras, and A. Zisserman, “Spot the conversation: Speaker diarisation in the wild,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH , vol. 2020, 2020, pp. 299–303

    2020

  18. [26]

    End-to-end audio-visual neural speaker diarization,

    M.-K. He, J. Du, and C.-H. Lee, “End-to-end audio-visual neural speaker diarization,” in Proc. Interspeech, 2022, pp. 1461–1465

    2022

  19. [27]

    Multi-input multi-output target-speaker voice activity detection for unified, flexible, and robust audio-visual speaker diarization,

    M. Cheng and M. Li, “Multi-input multi-output target-speaker voice activity detection for unified, flexible, and robust audio-visual speaker diarization,” arXiv preprint arXiv:2401.08052 , 2024

    2024 arXiv

  20. [28]

    Quality- aware end-to-end audio-visual neural speaker diarization,

    M.-K. He, J. Du, S.-T. Niu, Q.-F. Liu, and C.-H. Lee, “Quality- aware end-to-end audio-visual neural speaker diarization,”arXiv preprint arXiv:2410.22350, 2024

    2024 arXiv

  21. [29]

    Semi-supervised multi-channel speaker diarization with cross- channel attention,

    S. Wu, J. Du, M.-K. He, S. Niu, H. Chen, H. Tang, and C.-H. Lee, “Semi-supervised multi-channel speaker diarization with cross- channel attention,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2023, pp. 1–8

    2023

  22. [30]

    Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario,

    Y . Fu, L. Cheng, S. Lv, Y . Jv, Y . Kong, Z. Chen, Y . Hu, L. Xie, J. Wu, H. Bu et al., “Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario,” in Proc. Interspeech 2021 , 2021, pp. 3665–3669

    2021

  23. [31]

    Ava-avd: Audio-visual speaker diarization in the wild,

    E. Z. Xu, Z. Song, S. Tsutsui, C. Feng, M. Ye, and M. Z. Shou, “Ava-avd: Audio-visual speaker diarization in the wild,” in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 3838–3847

    2022

  24. [32]

    Semi-supervised training with pseudo-labeling for end- to-end neural diarization,

    Y . Takashima, Y . Fujita, S. Horiguchi, S. Watanabe, P. Garc ´ıa, and K. Nagamatsu, “Semi-supervised training with pseudo-labeling for end- to-end neural diarization,” in 22nd Annual Conference of the Inter- national Speech Communication Association, INTERSPEECH 2021 . Intern...

    2021

  25. [33]

    Chime-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,

    S. Watanabe, M. Mandel, J. Barker, E. Vincent, A. Arora, X. Chang, S. Khudanpur, V . Manohar, D. Povey, D. Rajet al., “Chime-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,” in CHiME 2020-6th International Workshop on Speech Processing in Ever...

    2020

  26. [34]

    The multimodal information based speech processing (misp) 2022 challenge: Audio- visual diarization and recognition,

    Z. Wang, S. Wu, H. Chen, M.-K. He, J. Du, C.-H. Lee, J. Chen, S. Watanabe, S. Siniscalchi, O. Scharenborg et al. , “The multimodal information based speech processing (misp) 2022 challenge: Audio- visual diarization and recognition,” in ICASSP 2023-2023 IEEE In- ternational Co...

    2022

  27. [35]

    Notsofar-1 challenge: New datasets, baseline, and tasks for distant meeting transcription,

    A. Vinnikov, A. Ivry, A. Hurvitz, I. Abramovski, S. Koubi, I. Gurvich, S. Pe’er, X. Xiao, B. M. Elizalde, N. Kanda et al., “Notsofar-1 challenge: New datasets, baseline, and tasks for distant meeting transcription,” CoRR, 2024

    2024

  28. [36]

    The nist speaker recognition evaluations: 1996-2001

    A. F. Martin and M. A. Przybocki, “The nist speaker recognition evaluations: 1996-2001.” in Odyssey, 2001, pp. 39–43

    1996

  29. [37]

    First dihard challenge evaluation plan,

    N. Ryant, K. Church, C. Cieri, A. Cristia, J. Du, S. Ganapathy, and M. Liberman, “First dihard challenge evaluation plan,” in tech. Rep. Linguistic Data Consortium, University of Pennsylvania, 2018

    2018

  30. [38]

    The second dihard diarization challenge: Dataset, task, and baselines,

    ——, “The second dihard diarization challenge: Dataset, task, and baselines,” arXiv preprint arXiv:1906.07839 , 2019

    1906 arXiv

  31. [39]

    The third dihard diarization challenge,

    N. Ryant, P. Singh, V . Krishnamohan, R. Varma, K. Church, C. Cieri, J. Du, S. Ganapathy, and M. Liberman, “The third dihard diarization challenge,” arXiv preprint arXiv:2012.01477 , 2020

    2012 arXiv

  32. [40]

    M2met: The icassp 2022 multi- channel multi-party meeting transcription challenge,

    F. Yu, S. Zhang, Y . Fu, L. Xie, S. Zheng, Z. Du, W. Huang, P. Guo, Z. Yan, B. Ma, X. Xu, and H. Bu, “M2met: The icassp 2022 multi- channel multi-party meeting transcription challenge,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Process...

    2022

  33. [41]

    The ami meeting corpus,

    W. Kraaij, T. Hain, M. Lincoln, and W. Post, “The ami meeting corpus,” in Proc. International Conference on Methods and Techniques in Behavioral Research , 2005, pp. 1–4

    2005

  34. [42]

    The multimodal information based speech processing (misp) 2025 challenge: Audio-visual diarization and recognition,

    M. Gao, S. Wu, H. Chen, J. Du, C.-H. Lee, S. Watanabe, J. Chen, S. S. Marco, and O. Scharenborg, “The multimodal information based speech processing (misp) 2025 challenge: Audio-visual diarization and recognition,” arXiv preprint arXiv:2505.13971 , 2025

    2025 arXiv

  35. [43]

    Msdwild: Multi-modal speaker diarization dataset in the wild

    T. Liu, S. Fan, X. Xiang, H. Song, S. Lin, J. Sun, T. Han, S. Chen, B. Yao, S. Liu et al., “Msdwild: Multi-modal speaker diarization dataset in the wild.” in INTERSPEECH, 2022, pp. 1476–1480

    2022

  36. [44]

    An approach to scene change detection,

    I. Bieda, A. Kisil, and T. Panchenko, “An approach to scene change detection,” in 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 1. IEEE, 2021, pp. 489–493

    2021

  37. [45]

    Dnsmos p. 835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “Dnsmos p. 835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” in ICASSP 2022-2022 IEEE international conference on 8 acoustics, speech and signal processing (ICASSP) . IEEE, 2022, pp. 886–890

    2022

  38. [46]

    Md-vqa: Multi-dimensional quality assessment for ugc live videos,

    Z. Zhang, W. Wu, W. Sun, D. Tu, W. Lu, X. Min, Y . Chen, and G. Zhai, “Md-vqa: Multi-dimensional quality assessment for ugc live videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1746–1755

    2023

  39. [47]

    Out of time: automated lip sync in the wild,

    J. S. Chung and A. Zisserman, “Out of time: automated lip sync in the wild,” in Computer Vision–ACCV 2016 Workshops: ACCV 2016 In- ternational Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13 . Springer, 2017, pp. 251–263

    2016

  40. [48]

    Retinaface: Single-shot multi-level face localisation in the wild,

    J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, “Retinaface: Single-shot multi-level face localisation in the wild,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 5203–5212

    2020

  41. [49]

    Simple online and realtime tracking with a deep association metric,

    N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in 2017 IEEE international conference on image processing (ICIP) . IEEE, 2017, pp. 3645–3649

    2017

  42. [50]

    Mediapipe: A framework for building perception pipelines,

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee et al. , “Mediapipe: A framework for building perception pipelines,” arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  43. [51]

    3d-speaker-toolkit: An open-source toolkit for multimodal speaker verification and diarization,

    Y . Chen, S. Zheng, H. Wang, L. Cheng, T. Zhu, R. Huang, C. Deng, Q. Chen, S. Zhang, W. Wang et al., “3d-speaker-toolkit: An open-source toolkit for multimodal speaker verification and diarization,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Sig...

    2025

  44. [52]

    Dover-lap: A method for combining overlap- aware diarization outputs,

    D. Raj, L. P. Garcia-Perera, Z. Huang, S. Watanabe, D. Povey, A. Stol- cke, and S. Khudanpur, “Dover-lap: A method for combining overlap- aware diarization outputs,” in 2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 881–888

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.