Pith. sign in

REVIEW 4 major objections 6 minor 50 references

Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An arrival-order speaker cache turns the offline Sortformer diarizer into a streaming system that reports state-of-the-art online diarization error at about one second of latency.

desk verdict A novel cache design and honest reporting make this a solid streaming diarization paper, but the 1 s SOTA claim only holds with a dataset-tuned post-processor that baselines don't get. read the letter →

arxiv 2507.18446 v1 pith:CKNU4CWV submitted 2025-07-24 eess.AS cs.SD

Ivan Medennikov , Taejin Park , Weiqing Wang , He Huang , Kunal Dhawan , Jinhan Wang , Jagadeesh Balam , Boris Ginsburg This is my paper · ORCID
classification eess.AScs.SD
keywords streamingspeakerdiarizationarrival-timeorderingcacheSortformeronlinelow-latencytracking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the offline Sortformer diarization model can be made to run online by adding a speaker cache that stores frame-level acoustic embeddings in speaker-arrival order. The resulting system, Streaming Sortformer with the Arrival-Order Speaker Cache (AOSC), reports diarization error rates below or comparable to published online systems at roughly one-second latency, and stays competitive at 0.32 seconds. If correct, this means online diarization does not need self-attention attractors or explicit permutation resolution; arrival-time ordering plus a scored frame cache is enough. That matters for live captioning, meetings, and multi-talker speech processing, where speaker labels are needed as audio arrives.

What carries the argument

The load-bearing object is the Arrival-Order Speaker Cache (AOSC), a fixed-length memory holding up to $M$ frame embeddings produced by the NEST pre-encoder, with $M=188$ frames in the reported runs. At each update it scores frames by $S_i = \log P_i + \sum_{j\neq i}\log(1-P_j)$, sets non-speech frames ($P_i<0.5$) to $-\infty$, adds a recency boost $\delta=0.05$, boosts $K=33$ frames per speaker by $-2\log 0.5$ and $K=66$ frames by $-\log 0.5$, appends three average-silence embeddings per speaker, and keeps the $M$ highest-scoring embeddings in speaker-index (arrival-time) order. This preserves a compact acoustic profile for each speaker without any trained attractor, so chunk-to-chunk speaker order is resolved by Sortformer's own arrival-time output instead of a permutation search. A FIFO queue of past chunks and a limited right context set the operating latency.

What would settle it

Take the released model and re-tune its cache and post-processing parameters from scratch on the development splits, then evaluate once on DIHARD III Eval; if the best DER after re-tuning is materially worse than 13.32%, the reported result depended on the specific hand-set configuration rather than on the mechanism itself.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a speaker cache whose entries are ordered by speaker index, matching the arrival-time order Sortformer is trained to output, lets an offline Sortformer run chunk-by-chunk without any permutation-matching step. The cache stores NEST encoder embeddings selected by per-frame scores, with silence embeddings appended, and is updated as frames are pushed out of a FIFO queue. After fine-tuning with this cache, one model achieves 13.32% DER on DIHARD III Eval at 1.04 seconds latency with post-processing, 10.79% on CALLHOME Part 2, and 5.09% on the CH109 two-speaker subset, while remaining usable at 0.32 seconds latency. Applying the same cache to the offline model without fine-tuning degrades results sharply, so the fine-tuning step is load-bearing.

Load-bearing premise

The load-bearing premise is that the hand-set cache update parameters (three silence frames, a recency boost of 0.05, and per-speaker quotas of 33 and 66 boosted frames) and the separately tuned post-processing thresholds generalize across datasets and latency settings without re-tuning.

Editorial extensions

If this is right

  • A single model, fine-tuned once and evaluated on all datasets without per-dataset tuning, reports 13.32% DER on DIHARD III Eval at 1.04 seconds latency with post-processing, below the 13.96% reported for the LS-EEND streaming baseline.
  • At 0.32 seconds latency the same model remains within about 0.1 to 1.4 absolute DER points of its one-second performance across the evaluated sets, so the accuracy-latency trade-off is shallow.
  • The streaming system can beat the offline Sortformer on long recordings and on 4-plus-speaker subsets because its fixed inference window avoids the offline model's long-sequence mismatch.
  • The cache design removes the permutation-resolution step that speaker-tracing buffers require, simplifying the online pipeline to a cache update plus one forward pass per chunk.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because AOSC is a discrete scored-frame memory, a natural next test is replacing the hand-set update rules (silence count, recency boost, per-speaker quotas) with learned score functions and checking whether DER stays near the reported values without development-set tuning.
  • Beyond the paper: comparing AOSC against a soft attention memory of the same size at equal latency would separate the value of storing exact past frames from the value of arrival-order structure itself.
  • Beyond the paper: if the per-speaker quota and silence padding are what preserve speaker identity across long pauses, the same mechanism may transfer to more than four speakers by enlarging the cache and the output dimension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a streaming extension of the Sortformer speaker diarization model. The key idea is an Arrival-Order Speaker Cache (AOSC) that stores frame-level embeddings from a NEST pre-encoder, orders them by speaker index according to arrival time, and updates itself by selecting frames with high model-based confidence scores. This avoids explicit inter-chunk permutation resolution. The authors fine-tune the offline Sortformer with AOSC, use a FIFO queue for context, and evaluate on DIHARD III Eval, CALLHOME Part2, and CH109 at latencies of 10 s, 1.04 s, and 0.32 s, reporting DER with and without a six-operation timestamp post-processor. They claim state-of-the-art results at about 1 s latency, with 13.32% DER on DIHARD III Eval (≤4 spk) at 1.04 s, and competitive results at 0.32 s.

Significance. If the comparison is confirmed, the paper is a useful contribution: it shows that a cache-based streaming Sortformer without attractors or explicit permutation resolution can match or exceed attractor-based online EEND systems at about 1 s latency, with an RTF of 0.093 on an RTX 6000 Ada. The method is described in enough detail to reimplement, the model and code are publicly available, and the evaluation spans three datasets and three latency settings. The score-based cache update with arrival-time ordering is a simple and plausible mechanism. However, the headline SOTA claim currently rests on an asymmetric post-processing comparison and on single-run DERs; with the evidence as presented, model-level superiority over LS-EEND is not yet established.

major comments (4)
  1. [Table 1, §4.3] The state-of-the-art claim is not apples-to-apples. The proposed system's about-1-s results include a six-operation timestamp post-processor whose parameters were tuned on DIHARD III Dev and CALLHOME Part1, while the published baselines (e.g., LS-EEND) are quoted without this post-processing. On DIHARD III Eval ≤4 spk, Streaming Sortformer-AOSC gives 14.57% DER without post-processing and 13.32% with it, so without the post-processor it loses to LS-EEND's 13.96%. The same pattern appears on CALLHOME Part2 2 spk (7.35% without post-processing vs. LS-EEND's 7.03%). The authors must either apply the identical post-processing to the baselines, report post-processed baseline numbers, or restrict the SOTA claim to raw model outputs, and in any case should provide an explicit ablation separating raw and post-processed DER for every row.
  2. [Table 1] All DERs are single-run point estimates without error bars, confidence intervals, or significance tests. The reported margins over baselines are small (about 0.6–1.2 DER) and could easily be within run-to-run variance or evaluation noise. I would like to see multiple decoding runs, or at minimum session-level bootstrap confidence intervals, and a matched-pair comparison between the proposed system and LS-EEND on the same evaluation sessions.
  3. [§3.2, §4.2] The core contribution, the AOSC update, depends on several hand-set hyperparameters (A=3, δ=0.05, K=33/66, Δ=−2log0.5/−log0.5) and on the score formula in Eq. (10), but the paper provides no ablation or sensitivity analysis. Without such an analysis it is impossible to know whether the reported gains are robust or an artifact of these specific values. Please add a sensitivity sweep for A, δ, and K/Δ, and compare Eq. (10) against simpler alternatives such as using the speaker posterior Pi alone or the model's raw output for each frame.
  4. [§4.1, §4.3] There is a domain-overfitting risk in the post-processing tuning: DIHARD III Dev and CALLHOME Part1 are included in the training set, and the post-processor is tuned on those exact splits. Since the post-processor contributes up to about 1.2 DER of improvement, the authors should demonstrate that the tuned parameters generalize, for example by reporting sensitivity of the final DER to the post-processing thresholds or by tuning on a held-out split rather than the same split used in training.
minor comments (6)
  1. [§3.2, Eq. (10)] The silence detection condition ('frames where Sortformer assigns a low probability to all speakers') is not defined numerically; please specify the threshold used to compute the average silence embedding.
  2. [Table 2] The table caption does not name the individual columns; add explicit column headers and clarify that 'Update Period' is the period after which AOSC updates are performed, as described in §3.3.
  3. [Table 1] The label 'Offline Sortformer-AOSC' is confusing when combined with latency values of 10 s and 1.04 s, since an offline system has no streaming latency; please clarify whether this variant processes audio chunk-wise at inference without fine-tuning.
  4. [References] References [16] and [17] are both assigned the same arXiv identifier (2411.13849); one of them appears to be incorrect.
  5. [§1] The sentence 'Model 1 and code are publicly available' appears to contain a stray footnote marker; it should read 'Model and code are publicly available'.
  6. [§4.1] For CH109, the statement that overlap with CALLHOME Part1 is 'very minor' should be quantified (e.g., number of sessions or duration of overlap) so readers can assess potential test-set contamination.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the streaming extension is evaluated against external benchmarks and the fitted components are disclosed hyperparameter/post-processing choices, not inputs that determine the claimed outputs.

full rationale

The paper's contribution is a streaming inference/training mechanism (AOSC) built on the prior Sortformer architecture. The load-bearing performance claim is an empirical comparison on DIHARD III, CALLHOME Part2, and CH109 against published streaming baselines. The AOSC update uses the model's own frame scores to decide which embeddings to cache; this is an algorithmic feedback loop during inference/training, not a derivation in which an output is defined as an input. The AOSC hyperparameters (A=3, delta=0.05, K=33/66) are hand-set, and the six-operation timestamp post-processor is tuned separately on DIHARD III Dev and CALLHOME Part1 for the corresponding eval sets. This is disclosed in the text and is standard hyperparameter tuning on development data; the reported DERs are measured on held-out eval partitions, so the eval numbers are not forced by construction. The paper also admits a minor overlap between CH109 and CALLHOME Part1, a data-contamination concern, not a circularity. The only self-citations are to the authors' prior Sortformer and NEST encoders; these are accompanied by public model/code releases and are used as architectural building blocks rather than as an unverified premise that uniquely forces the conclusions. No equation is shown to be definitionally equivalent to another, and no fitted parameter is renamed as a prediction. Therefore no circular step can be exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The central performance claim rests on several hand-set cache and post-processing hyperparameters, on the Sortformer arrival-ordering property, and on the transferability of fine-tuning from one configuration to several inference settings. No new physical or mathematical entities are introduced.

free parameters (6)
  • A (silence embeddings per speaker) = 3
    Number of average silence embeddings appended per speaker in the cache; chosen by hand, no ablation.
  • delta (recency boost) = 0.05
    Score boost added to newly added frames to favor recent data; chosen by hand, no sensitivity analysis.
  • K_strong and K_weak (per-speaker frame quotas) = 33 and 66
    Number of highest-scoring frames per speaker boosted to guarantee representation; chosen by hand.
  • Delta (boosting strength) = -2 log 0.5 and -log 0.5
    Strong and weak boost amplitudes used in step 5 of the cache update; chosen by hand.
  • Post-processing parameters = Not reported; tuned on DIHARD Dev and CALLHOME Part1
    Two sets of onset/offset thresholds, padding, and min-duration values tuned per benchmark dev split, then applied to eval; values not listed.
  • Right context limit during training = 7 frames (560 ms) with 50% probability
    Randomly limits right self-attention context to reduce future dependence; a training-time hyperparameter.
assumptions (4)
  • domain assumption Sort Loss enforces stable arrival-time ordering of output speakers.
    Section 2.1: the streaming method relies on this ordering persisting across chunks.
  • ad hoc to paper Randomly permuting the speaker cache during fine-tuning makes the model robust to cache ordering at inference.
    Section 4.2: no theoretical justification, only empirical.
  • ad hoc to paper The score formula in Eq. 10 is a reasonable measure of frame informativeness for speaker identity.
    Section 3.2: the log-sum-excluded formulation is introduced without derivation.
  • domain assumption Fine-tuning with 15-second windows and a 188-frame cache transfers to the various latency setups in Table 2.
    Section 4.2 and Table 2: training uses one configuration while inference uses three different chunk and cache configurations.
invented entities (1)
  • Average silence embedding
    purpose: Represents non-speech frames in the cache to help speaker transition detection.
    A computed average of frames where all speaker probabilities are low; it is a procedural artifact, not an independently verifiable entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering." pith.science (2026). https://pith.science/paper/CKNU4CWV

@misc{pith2026250718446,
  author       = {Pith},
  title        = {Pith review of: Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CKNU4CWV}},
  note         = {Machine review of arXiv:2507.18446}
}
read the original abstract

This paper presents a streaming extension for the Sortformer speaker diarization framework, whose key property is the arrival-time ordering of output speakers. The proposed approach employs an Arrival-Order Speaker Cache (AOSC) to store frame-level acoustic embeddings of previously observed speakers. Unlike conventional speaker-tracing buffers, AOSC orders embeddings by speaker index corresponding to their arrival time order, and is dynamically updated by selecting frames with the highest scores based on the model's past predictions. Notably, the number of stored embeddings per speaker is determined dynamically by the update mechanism, ensuring efficient cache utilization and precise speaker tracking. Experiments on benchmark datasets confirm the effectiveness and flexibility of our approach, even in low-latency setups. These results establish Streaming Sortformer as a robust solution for real-time multi-speaker tracking and a foundation for streaming multi-talker speech processing.

Figures

Figures reproduced from arXiv: 2507.18446 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Streaming steps with speaker cache update. The embeddings from NEST pre-encoder are stored in the speaker cache. embeddings in the order of speaker indices, which naturally corresponds to the arrival-time order of all previously observed speakers. While prior works [13, 14] also address permutation resolution using speaker appearance order, our method inher￾ently predicts speakers in arrival-time order without relyi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 46 canonical work pages

  1. [1]

    Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering

    Introduction As the accuracy of Automatic Speech Recognition (ASR) sys- tems continues to improve, the demand for robust speaker di- arization frameworks has grown significantly. This has spurred increasing interest in developing diarization systems capable of operating seamlessly in live, streaming environments. The ability to accurately tag speakers in ...

    work page Pith review arXiv 2025

  2. [2]

    Related Work 2.1. Foundation: Offline Sortformer The Sortformer model, as proposed in [ 18], consists of two primary components: a self-supervised pretrained NEST en- coder [19] based on the Fast-Conformer (FC) [20] architecture, and a stack of Transformer [21] encoder layers on top. The model outputs four sigmoids, allowing it to predict the activations ...

    work page

  3. [3]

    Proposed Method 3.1. Arrival-Order Speaker Cache While our approach also utilizes the STB concept, it introduces a specialized buffer design that aligns with Sortformer’s core idea and eliminates the need for an explicit permutation resolution step. More specifically, we propose the Arrival-Order Speaker Cache (AOSC), which stores frame-level embeddings f...

    work page

  4. [4]

    Compute speaker scores S for each frame: Si = log Pi + X j̸=i log(1 − Pj), (10) where i is the speaker index, and P represents the model’s prediction for the frame

    work page

  5. [5]

    Detect silence frames where the model assigns low prob- ability to all speakers, then compute the average silence embedding over these frames

    work page

  6. [6]

    Disable non-speech scores: if Pi < 0.5, set Si = −∞

    work page

  7. [7]

    Prioritize recent frames : For frames corresponding to newly added embeddings, increase their scores by δ >0 to favor keeping recent speaker data in the speaker cache

    work page

  8. [8]

    Ensure speaker representation : For each speaker, in- crease K highest scores by ∆ > 0, ensuring that each speaker is represented in the speaker cache

    work page

Show all 50 references
  1. [9]

    Append A scores of +∞ for each speaker, corresponding to the average silence embedding

  2. [10]

    For frames corre- sponding to +∞ or −∞ scores, use the average silence embedding instead

    Concatenate scores for all speakers, then select the M highest-scoring frames and return the corresponding em- beddings, while preserving their order. For frames corre- sponding to +∞ or −∞ scores, use the average silence embedding instead. The resulting sequence preserves emb...

  3. [11]

    Datasets We adopted the training dataset from [ 18], which consists of 5150 hours of simulated mixtures and 2030 hours of real multi- talker speech

    Experimental Results 4.1. Datasets We adopted the training dataset from [ 18], which consists of 5150 hours of simulated mixtures and 2030 hours of real multi- talker speech. The real speech data includes Fisher English Train- ing Speech Part 1 and 2 [22], the AMI Corpus Indiv...

    2000

  4. [12]

    Offline Sortformer: An offline system where the entire input audio is processed at once. 3We use two-fold splits from the Kaldicallhome_diarization recipe [ 29], where Part1 is used for training and fine-tuning, and Part2 is reserved for evaluation, consistent with the methodo...

  5. [13]

    Offline Sortformer-AOSC: A system that employs AOSC only during inference, without fine-tuning

  6. [14]

    The setup details for each latency configuration are provided in Table 2

    Streaming Sortformer-AOSC: A fully streaming system fine- tuned with the AOSC module to optimize performance. The setup details for each latency configuration are provided in Table 2. It should be noted that the declared latency values refer to the input buffer delay and do no...

  7. [15]

    Our speaker cache management technique dynamically adjusts the cache size for each speaker, focusing on speech frames that are most valuable for caching

    Conclusion In this paper, we propose a streaming version of the Sortformer diarization model, incorporating a novel speaker cache mecha- nism called AOSC. Our speaker cache management technique dynamically adjusts the cache size for each speaker, focusing on speech frames that...

  8. [16]

    End-to-end neural speaker diarization with permutation-free objectives,

    Y . Fujita, N. Kanda, S. Horiguchi, K. Nagamatsu, and S. Watan- abe, “End-to-end neural speaker diarization with permutation-free objectives,” in Proc. Interspeech 2019, 2019, pp. 4300–4304

    2019

  9. [17]

    End-to-end neural speaker diarization with self- attention,

    Y . Fujita, N. Kanda, S. Horiguchi, Y . Xue, K. Nagamatsu, and S. Watanabe, “End-to-end neural speaker diarization with self- attention,” in 2019 IEEE Automatic Speech Recognition and Un- derstanding Workshop (ASRU). IEEE, 2019, pp. 296–303

    2019

  10. [18]

    Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,

    D. Yu, M. Kolbæk, Z.-H. Tan, and J. Jensen, “Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2017, pp. 241–245

    2017

  11. [19]

    Neural speaker diarization with speaker-wise chain rule,

    Y . Fujita, S. Watanabe, S. Horiguchi, Y . Xue, J. Shi, and K. Naga- matsu, “Neural speaker diarization with speaker-wise chain rule,” arXiv preprint arXiv:2006.01796, 2020

    2006 arXiv

  12. [20]

    End-to-end speaker diarization conditioned on speech activity and overlap detection,

    Y . Takashima, Y . Fujita, S. Watanabe, S. Horiguchi, P. García, and K. Nagamatsu, “End-to-end speaker diarization conditioned on speech activity and overlap detection,” in 2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 849–856

    2021

  13. [21]

    End-to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors,

    S. Horiguchi, Y . Fujita, S. Watanabe, Y . Xue, and K. Nagamatsu, “End-to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors,” in Proc. Interspeech 2020, 2020, pp. 269–273

    2020

  14. [22]

    Encoder-decoder based attractors for end-to-end neural diariza- tion,

    S. Horiguchi, Y . Fujita, S. Watanabe, Y . Xue, and P. Garcia, “Encoder-decoder based attractors for end-to-end neural diariza- tion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 1493–1507, 2022

    2022

  15. [23]

    Online neural diarization of unlimited numbers of speakers using global and local attractors,

    S. Horiguchi, S. Watanabe, P. García, Y . Takashima, and Y . Kawaguchi, “Online neural diarization of unlimited numbers of speakers using global and local attractors,” IEEE/ACM Trans- actions on Audio, Speech, and Language Processing , vol. 31, pp. 706–720, 2022

    2022

  16. [24]

    Attention-based encoder- decoder end-to-end neural diarization with embedding enhancer,

    Z. Chen, B. Han, S. Wang, and Y . Qian, “Attention-based encoder- decoder end-to-end neural diarization with embedding enhancer,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 32, pp. 1636–1649, 2024

    2024

  17. [25]

    BW-EDA-EEND: Streaming end-to-end neural speaker diarization for a variable number of speakers,

    E. Han, C. Lee, and A. Stolcke, “BW-EDA-EEND: Streaming end-to-end neural speaker diarization for a variable number of speakers,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 7193–7197

    2021

  18. [26]

    Online streaming end-to-end neural diarization handling overlapping speech and flexible numbers of speakers,

    Y . Xue, S. Horiguchi, Y . Fujita, Y . Takashima, S. Watanabe, L. P. G. Perera, and K. Nagamatsu, “Online streaming end-to-end neural diarization handling overlapping speech and flexible numbers of speakers,” in Interspeech 2021, 2021, pp. 3116–3120

    2021

  19. [27]

    Online end-to-end neural diarization with speaker- tracing buffer,

    Y . Xue, S. Horiguchi, Y . Fujita, S. Watanabe, P. García, and K. Nagamatsu, “Online end-to-end neural diarization with speaker- tracing buffer,” in 2021 IEEE Spoken Language Technology Work- shop (SLT). IEEE, 2021, pp. 841–848

    2021

  20. [28]

    Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors,

    D. Liang, N. Shao, and X. Li, “Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 10 521–10 525

    2024

  21. [29]

    LS-EEND: Long-form streaming end-to-end neural diarization with online attractor extraction,

    D. Liang and X. Li, “LS-EEND: Long-form streaming end-to-end neural diarization with online attractor extraction,” arXiv preprint arXiv:2410.06670, 2024

    2024 arXiv

  22. [30]

    Target-speaker voice activity de- tection: a novel approach for multi-speaker diarization in a dinner party scenario,

    I. Medennikov, M. Korenevsky, T. Prisyach, Y . Khokhlov, M. Ko- renevskaya, I. Sorokin, T. Timofeeva, A. Mitrofanov, A. An- drusenko, I. Podluzhny et al., “Target-speaker voice activity de- tection: a novel approach for multi-speaker diarization in a dinner party scenario,” in...

    2020

  23. [32]

    Sequence-to-sequence neural diarization with automatic speaker detection and representation,

    M. Cheng, Y . Lin, and M. Li, “Sequence-to-sequence neural diarization with automatic speaker detection and representation,” arXiv preprint arXiv:2411.13849, 2024

    2024 arXiv

  24. [33]

    Sortformer: Seamless integration of speaker diarization and asr by bridging timestamps and tokens,

    T. Park, I. Medennikov, K. Dhawan, W. Wang, H. Huang, N. R. Koluguri, K. C. Puvvada, J. Balam, and B. Ginsburg, “Sortformer: Seamless integration of speaker diarization and asr by bridging timestamps and tokens,” arXiv preprint arXiv:2409.06656, 2024

    2024 arXiv

  25. [34]

    NEST: Self-supervised fast conformer as all-purpose seasoning to speech processing tasks,

    H. Huang, T. Park, K. Dhawan, I. Medennikov, K. C. Puvvada, N. R. Koluguri, W. Wang, J. Balam, and B. Ginsburg, “NEST: Self-supervised fast conformer as all-purpose seasoning to speech processing tasks,” in ICASSP 2025-2025 IEEE International Con- ference on Acoustics, Speech ...

    2025

  26. [35]

    Fast conformer with linearly scalable attention for efficient speech recognition,

    D. Rekesh, N. R. Koluguri, S. Kriman, S. Majumdar, V . Noroozi, H. Huang, O. Hrinchuk, K. Puvvada, A. Kumar, J. Balam et al., “Fast conformer with linearly scalable attention for efficient speech recognition,” in 2023 IEEE Automatic Speech Recognition and Understanding Worksho...

    2023

  27. [36]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  28. [37]

    The Fisher Corpus: A Re- source for the Next Generations of Speech-to-text,

    C. Cieri, D. Miller, and K. Walker, “The Fisher Corpus: A Re- source for the Next Generations of Speech-to-text,” inProc. LREC, 2004, pp. 69–71

    2004

  29. [38]

    The AMI corpus,

    University of Edinburgh, “The AMI corpus,” https://www.openslr. org/16/

  30. [39]

    Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks,

    F. Landini, J. Profant, M. Diez, and L. Burget, “Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks,” Computer Speech & Language, vol. 71, p. 101254, 2022

    2022

  31. [40]

    Third DIHARD challenge evaluation plan,

    N. Ryant, K. Church, C. Cieri, J. Du, S. Ganapathy, and M. Liber- man, “Third DIHARD challenge evaluation plan,” arXiv preprint arXiv:2006.05815, 2020

    2006 arXiv

  32. [41]

    Spot the conversation: speaker diarisation in the wild,

    J. S. Chung, J. Huh, A. Nagrani, T. Afouras, and A. Zisserman, “Spot the conversation: speaker diarisation in the wild,” in Proc. Interspeech 2020, 2020, pp. 299–303

    2020

  33. [42]

    The ICSI meeting corpus,

    University of Edinburgh, “The ICSI meeting corpus,” https:// groups.inf.ed.ac.uk/ami/icsi/

  34. [43]

    AISHELL-4: An open source dataset for speech enhancement, separation, recognition and speaker diariza- tion in conference scenario,

    Y . Fu, L. Cheng, S. Lv, Y . Jv, Y . Kong, Z. Chen, Y . Hu, L. Xie, J. Wu, H. Bu et al. , “AISHELL-4: An open source dataset for speech enhancement, separation, recognition and speaker diariza- tion in conference scenario,” in Proc. Interspeech 2021, 2021, pp. 3665–3669

    2021

  35. [44]

    Kaldi x-vector Recipe v2,

    Kaldi, “Kaldi x-vector Recipe v2,” https://github.com/kaldi-asr/ kaldi/tree/master/egs/callhome_diarization/v2

  36. [45]

    2000 NIST Speaker Recognition Evaluation,

    M. Przybocki and A. Martin, “2000 NIST Speaker Recognition Evaluation,” 2001. [Online]. Available: https://catalog.ldc.upenn. edu/LDC2001S97

    2000

  37. [46]

    M2MeT: The ICASSP 2022 multi-channel multi-party meeting transcription challenge,

    F. Yu, S. Zhang, Y . Fu, L. Xie, S. Zheng, Z. Du, W. Huang, P. Guo, Z. Yan, B. Ma, X. Xu, and H. Bu, “M2MeT: The ICASSP 2022 multi-channel multi-party meeting transcription challenge,” in Proc. ICASSP. IEEE, 2022

    2022

  38. [47]

    DiPCo–dinner party corpus,

    M. V . Segbroeck, A. Zaid, K. Kutsenko, C. Huerta, T. Nguyen, X. Luo, B. Hoffmeister, J. Trmal, M. Omologo, and R. Maas, “DiPCo–dinner party corpus,” in Proc. Interspeech 2020, 2020, pp. 434–436

    2020

  39. [48]

    STCON system for the CHiME-8 challenge,

    A. Mitrofanov, T. Prisyach, T. Timofeeva, S. Novoselov, M. Ko- renevsky, Y . Khokhlov, A. Akulov, A. Anikin, R. Khalili, I. Lezhenin et al., “STCON system for the CHiME-8 challenge,” in Proc. CHiME 2024, 2024, pp. 13–17

    2024

  40. [49]

    Callhome american english speech,

    A. Canavan, D. Graff, and G. Zipperlen, “Callhome american english speech,” Web Download, Philadelphia, 1997, lDC97S42. [Online]. Available: https://catalog.ldc.upenn.edu/LDC97S42

    1997

  41. [50]

    SpecAugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “SpecAugment: A simple data augmentation method for automatic speech recognition,” inProc. Interspeech 2019, 2019, pp. 2613–2617

    2019

  42. [51]

    A study on data augmentation of reverberant speech for robust speech recognition,

    T. Ko, V . Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2017, pp. 5220–5224

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.