Pith. sign in

REVIEW 4 major objections 4 minor 56 references

Identity-Faithful Audio-Visual Target Speaker Extraction with REAL-2MIX and VOXBLINK2-AVSE

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper establishes REAL-2MIX as a paired identity test for audio-visual target speaker extraction and reports a separator that achieves 82.22% strict output correctness on it.

desk verdict REAL-2MIX is a genuinely useful real-scene benchmark for audio-visual TSE, but the person-to-speaker mapping that defines its ground truth is under-audited; the model results are honestly caveated but not yet reproducible. read the letter →

arxiv 2608.03964 v2 pith:SANUBS7C submitted 2026-08-04 eess.AS

Peijun Yang , Zhan Jin , Xiaoyi Qin , Ruiyi Gan , Hao Wang , Juan Liu , Ming Li This is my paper
classification eess.AS
keywords audio-visualtargetspeakerextractionidentity-faithfulevaluationrealtwo-speakermixturesvisualbypasssimilarityoverlapdetectionMandarinspeechbenchmarktrainingcorpus
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a failure mode in audio-visual target speaker extraction: a separator can ignore the video and keep outputting the louder voice. It introduces REAL-2MIX, a benchmark of jointly recorded real Mandarin two-speaker mixtures with synchronized multi-view video, where every mixture is queried once with each speaker's lip cue, so the model must return speaker A under A's video and speaker B under B's video. To train on matched data, it curates VOXBLINK2-AVSE, 766 hours of synchronized audio–lip pairs from over 28,000 identities, and proposes an extractor that feeds frozen audio-visual HuBERT features into a TF-GridNet separator with layer-wise visual modulation. On the full test manifest the best checkpoint achieves 0.2261 character error rate, 82.22% strict correct identity outputs, and 69.53% of mixtures where both prompts are correct. The paper argues that identity plus residual-overlap metrics, not just reconstruction error, should be the benchmark for this task.

What carries the argument

The load-bearing object is the paired identity-faithful evaluation protocol built into REAL-2MIX: each Stage-III overlap clip is listed twice in the manifest, once per visual target, with preceding A-only and B-only stages supplying in-scene voice references; the success criterion is that $f_\theta(x, v_A) \approx s_A$ and $f_\theta(x, v_B) \approx s_B$. On the model side, the controlling mechanism is a frozen 1,280-dimensional projected AV-HuBERT visual stream that is injected both as a time-frequency conditioning map and as per-block FiLM scale/bias parameters in a six-block TF-GridNet separator, so the visual cue is reinforced throughout the network rather than used once at the input. The identity metric gates outputs by an overlap-speech detector before applying a speaker-margin comparison to scene-matched solo references.

What would settle it

Manually re-verify the person-to-speaker labels for all 77 scenes from the raw synchronized video and the headset-microphone solo stages, then recompute the 12,076-row CER and strict-identity numbers; if any scene's mapping disagrees with the manifest, the reported 82.22% and 69.53% are not clean measures of lip-cue following.

Watch

Extended reading notes

Core claim

The paper's central claim is that REAL-2MIX turns audio-visual TSE into a paired identity test: the same jointly recorded mixture must produce A under A's lip cue and B under B's lip cue, so visual bypass—where both prompts return the acoustically dominant voice—is directly penalized. The evaluation combines an overlap-detection gate with speaker similarity to in-scene solo references, defining 'strict correct' as an output that is single-speaker and closer to the prompted target than to the other speaker. The paper reports the best archived checkpoint at 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success over 6,038 mixtures and 12,076 target rows, and shows that matched training data and layer-wise FiLM conditioning matter more than enlarging the separator. It also claims VOXBLINK2-AVSE fills the training-data gap with quality-filtered, identity-diverse synchronized audio–lip pairs, while noting that the auxiliary speaker-margin loss needs a controlled rerun before its causal effect can be trusted.

Load-bearing premise

The evaluation depends on the person-to-speaker mapping in each REAL-2MIX scene being correct: if the video track assignments to the A and B transcript labels are swapped or wrong in even one scene, every A/B evaluation row for that scene is mislabeled and the headline identity scores are contaminated.

Editorial extensions

If this is right

  • If REAL-2MIX becomes the standard test, audio-visual TSE systems can be compared on jointly recorded real overlap without needing isolated clean sources, and visual bypass is directly scored.
  • The paired evaluation means a model that always picks the loudest voice will show zero speaker margin and near-zero both-output strict success, making cue-following visible.
  • Matched, identity-diverse training data (VOXBLINK2-AVSE) improves CER and strict correctness more than doubling GridNet capacity or adding width, so data-domain match should be prioritized.
  • Layer-wise FiLM is associated with a drop in residual overlap from 63.85% to 12.45%, suggesting that late-layer visual conditioning suppresses the interference channel, although the comparison is not fully controlled.
  • The 69.53% both-output strict success rate sets a concrete bar for later systems to beat on the complete manifest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the benchmark's headline numbers rest entirely on the manually reviewed person-to-speaker mapping; reporting the number of reviewed scenes and the mapping's estimated error rate would let users bound the label noise.
  • Because the paper notes that absolute OSD-first rates are threshold-dependent, a threshold sweep or calibration curve would make cross-system comparisons more reliable than a single fixed threshold.
  • The read-Mandarin-only corpus leaves open whether identity-faithful extraction generalizes to spontaneous, code-switched, or non-read speech; an extension with prompted conversational overlap would test this.
  • If the benchmark is reused for other languages or domains, adding a synthetic-mixture calibration set with known source identities would check that the OSD-first identity scoring tracks true extraction accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces REAL-2MIX, a 77-scene Mandarin audio-visual target speaker extraction benchmark of jointly recorded real two-speaker mixtures with preceding solo references and multi-view video, and VOXBLINK2-AVSE, a curated 250,828-pair training corpus from VoxBlink2. It also proposes an AV-HuBERT-conditioned TF-GridNet extractor that combines input-level visual fusion, layer-wise FiLM, and an auxiliary speaker-margin loss. On the full 6,038-mixture manifest, the best archived checkpoint is reported to achieve 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success. The central claims are that REAL-2MIX turns AV-TSE into a paired identity test and that the proposed extractor succeeds on that test.

Significance. REAL-2MIX addresses a genuine gap: existing AV-TSE benchmarks mostly mix independently recorded utterances, while real jointly recorded mixtures lack isolated ground-truth sources. The paired A/B query design is a useful way to expose visual bypass, and the use of external, independently trained ASR and speaker encoders plus OSD is a sensible non-intrusive evaluation framework. VOXBLINK2-AVSE is a substantial, identity-disjoint audio-lip corpus with explicit provenance keys. The paper is also commendably honest about several limitations: it states that the speaker-loss comparison is not causal, that the FiLM comparison is a diagnostic rather than a controlled ablation, that all rows are single checkpoints, and that the strongest checkpoint's launch CLI was not fully preserved. If the remaining measurement assumptions are tightened, the benchmark and model would be a valuable contribution.

major comments (4)
  1. [§3.4 and Appendix A] The person-to-speaker mapping that defines REAL-2MIX's ground truth is not audited to a publishable standard. The paper states that 52 videos had identity exchanges corrected and that bootstrap resampling was used to assess stability, but it never reports the number of low-confidence or manually reviewed scenes, the bootstrap failure rate, or a final mapping error rate. Because each scene contributes roughly 78 mixtures (156 of the 12,076 evaluation rows), a single systematically mislabeled scene shifts the headline both-strict rate by about 1.3 percentage points, and a swapped mapping would mislabel both the A-query and B-query rows for every mixture in that scene. The release should include the mapping audit, the manual-review list, and the identities of any corrected scenes so the paired-identity claim is independently verifiable.
  2. [§6.4] The OSD-first strict evaluation uses a mean overlap posterior threshold of 0.35 to classify outputs as residual overlap, and that threshold directly determines the headline strict correctness (82.22%) and both-strict (69.53%) numbers. The paper only asserts that the ordering is stable for thresholds from 0.25 to 0.45 and does not document how 0.35 was chosen; if it was selected on the test set, the absolute rates are optimistically biased. Please state whether the threshold was fixed a priori or tuned on a development set, and report the absolute rates across the stated range for every configuration in Tables 4 and 5.
  3. [§6.5, Tables 4–5] The main model conclusions are not backed by controlled experiments. The claimed FiLM benefit (63.85% vs. 12.45% residual overlap) comes from checkpoints that differ in training configuration, as the paper itself acknowledges; the speaker-loss comparison uses the checkpoint whose launch CLI "was not fully preserved"; and all reported rows are single checkpoints. Consequently, the abstract's best numbers are reproducible only as archived outputs, not as findings about the architecture. Please provide matched retraining runs with preserved CLI, multiple seeds, and confidence intervals, or explicitly restrict the abstract and conclusions to the diagnostic interpretation.
  4. [§4.2 and §4.3] The DNSMOS OVRL threshold of 3.3 defines VOXBLINK2-AVSE, but no downstream validation shows that this operating point is preferable to nearby thresholds such as 3.0 or 3.5. Because the corpus is one of the paper's three central contributions, the curation threshold should be supported either by a small training sweep or by an analysis showing that the retained set is better suited for AV-TSE training; otherwise the 766.17-hour scale claim is tied to an unvalidated free parameter.
minor comments (4)
  1. [§3.3] The sentence "After filtering there leaving 6,038 evaluable mixtures" is ungrammatical; it should read "After filtering, 6,038 evaluable mixtures remain."
  2. [§4.3] The sentence "Causing that we mainly using REAL-2MIX for testing, we don't dividing testing partition here" is ungrammatical and should be rewritten, for example as "Because we use REAL-2MIX for testing, we do not define a separate test partition in VOXBLINK2-AVSE."
  3. [Tables 4 and 5] The dagger symbol appears next to "None" in the Spk. loss column of Tables 4 and 5 but is never explained in the captions; please add a footnote describing what the dagger denotes.
  4. [§6.4] The term "OSD-single ID" would be clearer as "conditional identity accuracy given OSD-single classification," since it conditions on the OSD model's output rather than measuring a single unconditional accuracy.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark results rest on external encoders and held-out real mixtures, not on a reduction of predictions to inputs.

full rationale

REAL-2MIX is an empirical benchmark and measurement study rather than a derived theory. The paired A/B identity test is a task definition grounded in recorded solo stages and a face-tracking/alignment pipeline; the headline numbers (0.2261 CER, 82.22% strict correctness, 69.53% both-strict success) are measurements of a checkpoint on held-out real mixtures. The training objective (Eq. 4) optimizes SI-SNR and an ECAPA-TDNN speaker-margin loss on VOXBLINK2-AVSE pairs, while evaluation uses independent encoders (WeSpeaker ResNet34, Qwen3-ASR, WavLM OSD) that were not fitted to REAL-2MIX. The only overlapping-author citations, M2S-AVSR for the frozen visual projection and VoxBlink2 as the curation source, are component choices or data provenance rather than load-bearing justifications that define the result. No equation in the paper makes a predicted quantity equal to its input by construction, no uniqueness theorem is imported, and no fitted parameter is renamed as a prediction. The absence of a reported manual-review count for the person-to-speaker mapping is a data-quality and auditing limitation, not a circular dependency, and would be better addressed under benchmark validation than under circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a handful of hand-chosen thresholds (OSD 0.35, DNSMOS 3.3, Viterbi cost 0.35) and on the correctness of the authors' identity-alignment pipeline. No new physical entities are introduced. The training corpus and benchmark are new resources, but they are data, not fitted parameters.

free parameters (6)
  • OSD residual-overlap threshold = 0.35
    Section 6.4: outputs with mean overlap posterior above 0.35 are labeled residual overlap; the reported 82.22% strict correctness and 69.53% both-output success depend on this threshold.
  • DNSMOS OVRL threshold = 3.3
    Section 4.2: VOXBLINK2-AVSE retains utterances with DNSMOS OVRL at least 3.3; this curation choice shapes the training corpus and therefore all model results.
  • Speaker-margin loss margin m = 0.2
    Section 5.3: margin in the auxiliary speaker loss; used in the margin-loss checkpoints, not in the best checkpoint.
  • Speaker-margin loss weight lambda_spk = 0.1
    Section 5.3: weight of the auxiliary loss in the total training objective.
  • Viterbi transition cost = 0.35
    Appendix A, Eq. (7): cost for switching the keep-or-swap state during identity tracking; affects the scene-level identity labels.
  • Duplicate-detection cosine threshold = 0.92
    Appendix A: frames with two face embedding similarities above 0.92 are treated as duplicates or collisions and replaced; affects track quality.
assumptions (5)
  • domain assumption The recorded mixture is well approximated as x = sA + sB + n with additive target, interferer, and noise.
    Section 3.1 defines the task this way; in real recordings, reverberation and device effects are absorbed into n.
  • domain assumption The manually reviewed person-to-speaker mapping is correct for all 77 scenes.
    The benchmark's ground truth depends on this mapping; Section 3.4 reports manual review but no final error rate.
  • domain assumption Frozen AV-HuBERT visual features (1,280-dimensional) carry enough phonetic and identity information to condition target extraction.
    Section 5.1 adopts these features without fine-tuning; if the visual branch is uninformative, the model cannot follow the target cue.
  • domain assumption The evaluation models (WeSpeaker ResNet34, Qwen3-ASR, WavLM OSD) are valid and sufficiently accurate proxies for identity, content, and overlap.
    Sections 6.2-6.4 rely on these external models; the paper itself notes OSD errors and artifacts can affect the proxy.
  • ad hoc to paper The OSD posterior threshold at 0.35 separates residual overlap from single-speaker outputs.
    Section 6.4 sets this threshold; the reported sensitivity analysis covers 0.25-0.45 but does not justify 0.35 as the operating point.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identity-Faithful Audio-Visual Target Speaker Extraction with REAL-2MIX and VOXBLINK2-AVSE." pith.science (2026). https://pith.science/paper/SANUBS7C

@misc{pith2026260803964,
  author       = {Pith},
  title        = {Pith review of: Identity-Faithful Audio-Visual Target Speaker Extraction with REAL-2MIX and VOXBLINK2-AVSE},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SANUBS7C}},
  note         = {Machine review of arXiv:2608.03964}
}
read the original abstract

Audio-visual target speaker extraction should return the speaker indicated by the video, yet a separator can ignore the visual cue and repeatedly output the acoustically dominant voice. We introduce REAL-2MIX, a Mandarin AV-TSE benchmark of jointly recorded real two-speaker mixtures with synchronized multi-view video. Each scene also contains preceding A-only and B-only stages that provide in-scene speaker references. It contains 77 scenes and 7,598 clips (11.84 hours), including 6,042 dual-annotated mixtures. After processing there leave 6,038 evaluable mixtures and 12,076 target-speaker rows. We additionally curate VOXBLINK2-AVSE from VoxBlink2, comprising 250,828 synchronized audio--lip-ROI pairs from 28,421 identities and 766.17 hours of speech. Our extractor uses frozen, 1,280-dimensional projected AV-HuBERT features, target-conditioned training, and layer-wise feature modulation. We jointly evaluate content with Qwen3-ASR-1.7B CER and target identity with WeSpeaker ResNet34 plus Overlapped Speech Detection (OSD). On the complete manifest, the best archived checkpoint obtains 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success.

Figures

Figures reproduced from arXiv: 2608.03964 by the authors.

Figure 1
Figure 1. QIANGDA recording setup and a representative frontal-view frame. Panel (a) shows the camera geometry and microphone arrangement; panel (b) shows a 0.5-m example with the two speaker regions marked by dashed boxes. 3.2 Geometry and Acquisition [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed AV-HuBERT-conditioned TF-GridNet. The frozen visual branch conditions [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 3 canonical work pages

  1. [1]

    ACM Transactions on Graphics , volume =

    Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation , author =. ACM Transactions on Graphics , volume =. 2018 , doi =

    2018

  2. [2]

    2025 , publisher=

    Zeng, Bang and Li, Ming , journal=. 2025 , publisher=

    2025

  3. [3]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2021 , doi =

    2021

  4. [4]

    Interspeech , pages =

    Multimodal SpeakerBeam: Single Channel Target Speech Extraction with Audio-Visual Speaker Clues , author =. Interspeech , pages =. 2019 , doi =

    2019

  5. [5]

    2022 , doi =

    Pan, Zexu and Ge, Meng and Li, Haizhou , journal =. 2022 , doi =

    2022

  6. [6]

    2019 , publisher=

    IEEE Journal of Selected Topics in Signal Processing , volume=. 2019 , publisher=

    2019

  7. [7]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume =

    Conv-TasNet: Surpassing Ideal Time--Frequency Magnitude Masking for Speech Separation , author =. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume =. 2019 , doi =

    2019

  8. [8]

    Interspeech , pages =

    VoxCeleb: A Large-Scale Speaker Identification Dataset , author =. Interspeech , pages =. 2017 , doi =

    2017

Show all 56 references
  1. [9]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

    Lip Reading Sentences in the Wild , author =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =. 2017 , doi =

    2017

  2. [10]

    2018 , doi =

    Afouras, Triantafyllos and Chung, Joon Son and Zisserman, Andrew , publisher =. 2018 , doi =

    2018

  3. [11]

    , booktitle =

    Le Roux, Jonathan and Wisdom, Scott and Erdogan, Hakan and Hershey, John R. , booktitle =. 2019 , doi =

    2019

  4. [12]

    2017 , doi =

    Bu, Hui and Du, Jiayu and Na, Xingyu and Wu, Bengu and Zheng, Hao , booktitle =. 2017 , doi =

    2017

  5. [13]

    2023 , doi =

    Wang, Zhong-Qiu and Cornell, Samuele and Choi, Shukjae and Lee, Younglo and Kim, Byeong-Yeol and Watanabe, Shinji , journal =. 2023 , doi =

    2023

  6. [14]

    Scenario-Aware Audio-Visual

    Pan, Zexu and Wichern, Gordon and Masuyama, Yoshiki and Germain, Fran. Scenario-Aware Audio-Visual. IEEE Automatic Speech Recognition and Understanding Workshop , pages =. 2023 , doi =

    2023

  7. [15]

    , title =

    Levenshtein, Vladimir I. , title =. Soviet Physics Doklady , volume =

  8. [16]

    2018 , doi =

    Perez, Ethan and Strub, Florian and de Vries, Harm and Dumoulin, Vincent and Courville, Aaron , booktitle =. 2018 , doi =

    2018

  9. [17]

    2020 , doi =

    Desplanques, Brecht and Thienpondt, Jenthe and Demuynck, Kris , booktitle =. 2020 , doi =

    2020

  10. [18]

    2020 , doi =

    Deng, Jiankang and Guo, Jia and Ververas, Evangelos and Kotsia, Irene and Zafeiriou, Stefanos , booktitle =. 2020 , doi =

    2020

  11. [19]

    IEEE International Conference on Computer Vision , pages =

    How Far Are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , author =. IEEE International Conference on Computer Vision , pages =. 2017 , doi =

    2017

  12. [20]

    2015 , doi =

    Schroff, Florian and Kalenichenko, Dmitry and Philbin, James , booktitle =. 2015 , doi =

    2015

  13. [21]

    and Zisserman, Andrew , booktitle =

    Cao, Qiong and Shen, Li and Xie, Weidi and Parkhi, Omkar M. and Zisserman, Andrew , booktitle =. 2018 , doi =

    2018

  14. [22]

    IEEE Transactions on Information Theory , volume =

    Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , author =. IEEE Transactions on Information Theory , volume =. 1967 , doi =

    1967

  15. [23]

    ACM Multimedia , pages =

    Is Someone Speaking? Exploring Long-Term Temporal Features for Audio-Visual Active Speaker Detection , author =. ACM Multimedia , pages =. 2021 , doi =

    2021

  16. [24]

    The Annals of Statistics , volume =

    Bootstrap Methods: Another Look at the Jackknife , author =. The Annals of Statistics , volume =. 1979 , doi =

    1979

  17. [27]

    VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark , author=. Proc. Interspeech 2024 , pages=. 2024 , doi=

    2024

  18. [28]

    2026 , doi =

    Shi, Xian and Wang, Xiong and Guo, Zhifang and Wang, Yongqi and Zhang, Pei and Zhang, Xinyu and Guo, Zishan and Hao, Hongkun and Xi, Yu and Yang, Baosong and Xu, Jin and Zhou, Jingren and Lin, Junyang , publisher =. 2026 , doi =

    2026

  19. [29]

    2025 , pages =

    Li, Shaole and Wang, Shuai and Han, Jiangyu and Zhang, Ke and Wang, Wupeng and Li, Haizhou , booktitle =. 2025 , pages =

    2025

  20. [30]

    ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors , author=. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2021 , organization=

    2021

  21. [31]

    Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. 2018. https://doi.org/10.48550/arXiv.1809.00496 LRS3-TED : A large-scale dataset for visual speech recognition . arXiv preprint

    2018 doi arXiv

  22. [32]

    Adrian Bulat and Georgios Tzimiropoulos. 2017. https://doi.org/10.1109/ICCV.2017.116 How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks) . In IEEE International Conference on Computer Vision, pages 1021--1030

    2017 doi

  23. [33]

    Parkhi, and Andrew Zisserman

    Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. 2018. https://doi.org/10.1109/FG.2018.00020 VGGFace2 : A dataset for recognising faces across pose and age . In IEEE International Conference on Automatic Face & Gesture Recognition, pages 67--74

    2018

  24. [34]

    Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. 2017. https://doi.org/10.1109/CVPR.2017.367 Lip reading sentences in the wild . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3444--3453

    2017 doi

  25. [35]

    Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou. 2020. https://doi.org/10.1109/CVPR42600.2020.00525 RetinaFace : Single-shot multi-level face localisation in the wild . In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5202--5211

    2020

  26. [36]

    Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. https://doi.org/10.21437/Interspeech.2020-2650 ECAPA-TDNN : Emphasized channel attention, propagation and aggregation in TDNN based speaker verification . In Interspeech, pages 3830--3834

    2020 doi

  27. [37]

    Bradley Efron. 1979. https://doi.org/10.1214/aos/1176344552 Bootstrap methods: Another look at the jackknife . The Annals of Statistics, 7(1):1--26

    1979

  28. [38]

    Freeman, and Michael Rubinstein

    Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T. Freeman, and Michael Rubinstein. 2018. https://doi.org/10.1145/3197517.3201357 Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separatio...

    2018

  29. [39]

    Ruohan Gao and Kristen Grauman. 2021. https://doi.org/10.1109/CVPR46437.2021.01524 Visualvoice: Audio-visual speech separation with cross-modal consistency . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15490--15500

    2021

  30. [40]

    Levenshtein

    Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707--710

    1966

  31. [41]

    Shaole Li, Shuai Wang, Jiangyu Han, Ke Zhang, Wupeng Wang, and Haizhou Li. 2025. https://doi.org/10.21437/Interspeech.2025-2662 REAL-T : Real conversational mixtures for target speaker extraction . In Interspeech 2025, pages 1923--1927

    2025 doi

  32. [42]

    Yuke Lin, Ming Cheng, Fulin Zhang, Yingying Gao, Shilei Zhang, and Ming Li. 2024. https://doi.org/10.21437/Interspeech.2024-1490 Voxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark . In Proc. Interspeech 2024, pages 4263--4267

    2024 doi

  33. [43]

    Yi Luo and Nima Mesgarani. 2019. https://doi.org/10.1109/TASLP.2019.2915167 Conv-tasnet: Surpassing ideal time--frequency magnitude masking for speech separation . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(8):1256--1266

    2019

  34. [44]

    Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. https://doi.org/10.21437/Interspeech.2017-950 Voxceleb: A large-scale speaker identification dataset . In Interspeech, pages 2616--2620

    2017 doi

  35. [45]

    Tsubasa Ochiai, Marc Delcroix, Keisuke Kinoshita, Atsunori Ogawa, and Tomohiro Nakatani. 2019. https://doi.org/10.21437/Interspeech.2019-1513 Multimodal speakerbeam: Single channel target speech extraction with audio-visual speaker clues . In Interspeech, pages 2718--2722

    2019 doi

  36. [46]

    Zexu Pan, Meng Ge, and Haizhou Li. 2022. https://doi.org/10.1109/TASLP.2022.3205759 USEV : Universal speaker extraction with visual cue . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:3032--3045

    2022

  37. [47]

    Germain, Sameer Khurana, Chiori Hori, and Jonathan Le Roux

    Zexu Pan, Gordon Wichern, Yoshiki Masuyama, Fran c ois G. Germain, Sameer Khurana, Chiori Hori, and Jonathan Le Roux. 2023. https://doi.org/10.1109/ASRU57964.2023.10389618 Scenario-aware audio-visual TF-GridNet for target speech extraction . In IEEE Automatic Speech Recognitio...

    2023

  38. [48]

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. 2018. https://doi.org/10.1609/aaai.v32i1.11671 FiLM : Visual reasoning with a general conditioning layer . In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32

    2018 doi

  39. [49]

    Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2021. https://doi.org/10.1109/ICASSP39728.2021.9414878 Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors . In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and...

    2021

  40. [50]

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. https://doi.org/10.1109/CVPR.2015.7298682 FaceNet : A unified embedding for face recognition and clustering . In IEEE Conference on Computer Vision and Pattern Recognition, pages 815--823

    2015

  41. [51]

    Bowen Shi, Wei-Ning Hsu, Kushal Lakhotia, and Abdelrahman Mohamed. 2022. https://doi.org/10.48550/arXiv.2201.02184 Learning audio-visual speech representation by masked multimodal cluster prediction . In International Conference on Learning Representations

    2022 doi arXiv

  42. [52]

    Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, and Junyang Lin. 2026. https://doi.org/10.48550/arXiv.2601.21337 Qwen3-ASR technical report . arXiv preprint

    2026 doi arXiv

  43. [53]

    Fei Su, Cancan Li, Ming Li, and Juan Liu. 2026. https://doi.org/10.48550/arXiv.2606.05763 M2s-avsr: Modality-aware multi-view self-supervised representation for robust audio-visual speech recognition . arXiv preprint arXiv:2606.05763

    2026 doi arXiv

  44. [54]

    Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, and Haizhou Li. 2021. https://doi.org/10.1145/3474085.3475587 Is someone speaking? exploring long-term temporal features for audio-visual active speaker detection . In ACM Multimedia, pages 3927--3935

    2021

  45. [55]

    Andrew J. Viterbi. 1967. https://doi.org/10.1109/TIT.1967.1054010 Error bounds for convolutional codes and an asymptotically optimum decoding algorithm . IEEE Transactions on Information Theory, 13(2):260--269

    1967

  46. [56]

    Zhong-Qiu Wang, Samuele Cornell, Shukjae Choi, Younglo Lee, Byeong-Yeol Kim, and Shinji Watanabe. 2023. https://doi.org/10.1109/TASLP.2023.3304482 TF-GridNet : Integrating full- and sub-band modeling for speech separation . IEEE/ACM Transactions on Audio, Speech, and Language ...

    2023

  47. [57]

    Bang Zeng and Ming Li. 2025. https://doi.org/10.1109/TASLPRO.2025.3572756 USEF-TSE : Universal speaker embedding free target speaker extraction . IEEE Transactions on Audio, Speech and Language Processing, 33:2110--2124

    2025

  48. [58]

    Kate r ina Z mol \' kov \'a , Marc Delcroix, Keisuke Kinoshita, Tsubasa Ochiai, Tomohiro Nakatani, Luk \'a s Burget, and Jan C ernock \`y . 2019. https://doi.org/10.1109/JSTSP.2019.2922820 SpeakerBeam : Speaker aware neural network for target speaker extraction in speech mixtu...

    2019

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.