Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videos

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fusing frozen CLAP audio and OWL-ViT video embeddings into a Cross-Modal Conformer roughly doubles detection F1 and halves direction error for stereo sound-event localization, versus challenge baselines.

desk verdict A useful stereo SELD system report with a credible architecture, but model selection on the test partition inflates the headline margin. read the letter →

arxiv 2507.04845 v1 pith:XP4GBOIS submitted 2025-07-07 eess.AS cs.LGeess.IVeess.SP

Davide Berghi , Philip J. B. Jackson This is my paper · ORCID
classification eess.AScs.LGeess.IVeess.SP
keywords SoundEventLocalizationandDetectionStereoaudioCross-ModalConformerCLAPOWL-ViTInter-channelleveldifferenceAutocorrelationfeaturesAudio-visualfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a major bottleneck in sound event localization and detection (SELD) on ordinary stereo video is semantic: spatial channels say where a sound comes from, but identifying what is making it and whether it is visible is what separates strong systems from weak ones. To test this, the authors fuse frozen, language-aligned semantic embeddings — CLAP for audio and OWL-ViT for video — into a standard CNN-Conformer SELD backbone using a Cross-Modal Conformer that lets one modality attend to another. Pretraining on large synthetic stereo scenes, augmenting by swapping left and right channels, and adding autocorrelation-based features for distance, their systems roughly double detection F1 and cut direction-of-arrival error roughly in half relative to the challenge baselines on the development set. The strongest audio-visual ensemble reaches F1=48.0% and a mean direction-of-arrival error of 14.0°, against an audio-visual baseline of F1=26.8% and 23.8°. If the approach holds, large pretrained semantic models can be plugged into SELD without multichannel microphone arrays, which matters for consumer video and media production.

What carries the argument

The central mechanism is the Cross-Modal Conformer (CMC), an adaptation of the Conformer (a convolution-augmented transformer) in which two feed-forward modules process two modalities in parallel and multi-head self-attention is replaced by multi-head cross-attention, with queries from one modality and keys and values from the other. The audio branch uses frozen CLAP embeddings as semantic keys and values for the SELD encoder's output, while the video branch uses frozen OWL-ViT patch tokens as keys and values for the combined audio representation, deliberately preserving the spatial layout of image patches rather than average-pooling it away. Stereo spatial cues are carried by inter-channel level differences (ILD) computed from the two channels' STFT magnitudes, alongside per-channel log-mel spectrograms; short-term power of the autocorrelation (stpACC) features, which encode reverberation-related cues, are added to support distance estimation. Training uses the auxiliary duplicating permutation invariant training (ADPIT) loss for up to three tracks, with synthetic FOA scenes generated by SpatialScaper, synthetic videos from SELDVisualSynth, and left-right audio channel swapping plus video flipping as augmentation.

What would settle it

A direct falsifier is to measure the cross-correlation peak delay between the left and right channels on clips with known source azimuth: a systematic nonzero lag would contradict the paper's premise that the stereo channels carry no time or phase differences, and would undermine the ILD-only spatial feature design.

Watch

Extended reading notes

Core claim

The paper's central claim is that semantic information from contrastive language-aligned models can be injected into a standard CNN-Conformer SELD backbone through a Cross-Modal Conformer, and that this combination — together with synthetic pre-training, left-right channel and video swap augmentation, and autocorrelation-based distance features — yields the reported gains on stereo SELD in regular video. On the development set, the audio-only systems reach F1 scores of 45.7% and 46.0% with direction-of-arrival errors of 15.0° and 15.2°, compared with an audio-only baseline of F1=22.8% and DOAE=24.5°. The audio-visual ensemble reaches F1=48.0%, DOAE=14.0°, on/off-screen accuracy of 80.8%, and a relative distance error of 29.3%, compared with an audio-visual baseline of F1=26.8%, DOAE=23.8%, and RDE=40.0%. The paper further claims that the roughly ten-percentage-point improvement in relative distance error is partly due to the stpACC autocorrelation features, and that human-keypoint post-processing improves only on/off-screen classification, not localization.

Load-bearing premise

The load-bearing premise is that the stereo channels in the dataset are arithmetically derived from first-order ambisonics and therefore contain no inter-channel time or phase differences, so the model can safely rely on inter-channel level differences as its only spatial cue.

Editorial extensions

If this is right

  • A SELD system can use frozen, pre-trained contrastive language-aligned encoders as a source of semantic information, avoiding the need to train those encoders on multichannel audio.
  • On stereo-only video input, both the audio-only and audio-visual systems roughly double F1 and reduce direction-of-arrival error by about ten degrees relative to the challenge baselines on the development set.
  • Combining four systems by an agreement-based ensemble, with per-class exceptions for Bell and Knock, adds further gains: the ensemble reaches F1=48.0% and DOAE=14.0°.
  • Autocorrelation-based stpACC features are credited with part of the roughly ten-percentage-point improvement in relative distance error.
  • Human-keypoint post-processing did not improve localization and was used only to correct on/off-screen labels, indicating a boundary of visual semantic refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the report leaves implicit is that the same cross-modal fusion recipe should transfer to other low-channel-count settings, such as monophonic consumer video or binaural recordings, where frozen semantic embeddings could compensate for reduced spatial information; this is testable by replacing the stereo front-end while keeping the CMC and encoders fixed.
  • Because the paper does not ablate CLAP, OWL-ViT, and stpACC separately, the relative contribution of each component is unknown; a natural follow-up is to remove each branch and measure F1, DOAE, and RDE on the same development partition.
  • The keypoint post-processing result suggests visual semantics are more reliably used as a prior for what is visible than as a correction to acoustic localization; one could test this by feeding keypoint-object association scores into the Cross-Modal Conformer rather than applying them after the fact.
  • Since the stereo channels are arithmetically derived from FOA, a system built on ILD may not generalize to true binaural recordings with physical microphone spacing; a domain-transfer experiment on binaural data would reveal how much of the gain depends on the derived-stereo premise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript reports systems submitted to the audio-only and audio-visual tracks of DCASE 2025 Task 3, which addresses stereo sound event localization and detection in regular video content. The proposed architecture combines a CNN-Conformer SELD encoder with frozen CLAP audio embeddings and OWL-ViT visual embeddings, fused through a Cross-Modal Conformer module. Acoustic input features are log-mel spectrograms, inter-channel level differences, and stpACC autocorrelation features. The models are pre-trained on synthetic audio and audio-visual datasets, fine-tuned on the stereo STARSS23 development set, and further improved through left-right channel swapping, a keypoint-based visual post-processing step, and an ensemble of four systems. On the development set, the submitted systems reach F1 scores of 45.7-48.0% and DOAE of 14.0-15.6 degrees, compared with baseline F1 values of 22.8% (audio-only) and 26.8% (audio-visual) and baseline DOAE values of 24.5 and 23.8 degrees, respectively.

Significance. If the reported margins survive an unbiased evaluation protocol, the paper would demonstrate a meaningful advance in stereo SELD for conventional frontal video, particularly in leveraging large-scale pre-trained language-aligned models to overcome the data-scarcity constraints of multichannel SELD. The synthetic data generation pipeline, the detailed Cross-Modal Conformer design, and the clear reporting of negative results (e.g., keypoint-based DOA correction degrading accuracy) are useful contributions, and the authors provide a public data link. However, the headline comparison is based on model selection on the evaluation partition and lacks ablations, so the evidence for the central claim of superiority is currently weaker than the abstract suggests. The paper is a well-structured technical report, but it needs additional experimental rigor to support a journal-level claim.

major comments (3)
  1. [Sec. 5.1 / Table 1] The evaluation protocol selects the best epoch by the highest F1 on the test partition of the development set, as stated in Sec. 5.1 ('selecting the best based on the highest F1 score on the test partition'). Table 1 therefore reports the optimistically selected result of an 80-epoch run, while the challenge baselines are fixed models evaluated once. This selection-on-the-evaluation-set protocol biases the comparison in favor of the submitted systems, and no confidence intervals or significance tests are provided. The abstract's claim that the systems 'substantially outperform' the baselines is thus not supported by an unbiased estimate; please report results under validation-based model selection or provide confidence intervals and a paired significance analysis.
  2. [Sec. 5.4] The 10-point RDE improvement over baselines is attributed 'in part to the inclusion of stpACC features', but no ablation removes stpACC or replaces it with a baseline feature. Similarly, the core claim that semantic embeddings from CLAP and OWL-ViT are responsible for the gains is not tested: the submitted systems also differ from the baselines by synthetic pretraining, CMC fusion, the weighted on/off-screen loss, and ensembling. Without ablations, the title and abstract's attribution of the results to spatial and semantic embedding integration is over-claimed. Please add at least a feature ablation (with/without stpACC) and a modality ablation (with/without CLAP and OWL-ViT branches).
  3. [Sec. 3] The acoustic feature design rests on the assumption that the stereo channels are 'arithmetically derived from FOA signals' and therefore have no inter-channel time or phase differences, so ITD is discarded and ILD is used as the primary spatial cue. This assumption is load-bearing but unverified. If the stereo generation pipeline introduces even small channel delays, or if the assumption fails for other stereo content, the model would be unable to exploit potentially useful ITD cues. Please provide a quantitative verification, e.g., a cross-correlation analysis of the left and right channels in the development set, or a precise description of the stereo generation code.
minor comments (5)
  1. [Sec. 2.1] The sentence 'Tin is chosen so that Tin/16 matches the label frame rate' is unclear; for 5-second clips at 10 labels per second, please state the exact input dimensions and how the temporal pooling aligns with the label rate.
  2. [Eq. (1)] The phrase 'where m in the mel frequency index' should read 'where m is the mel frequency index.'
  3. [Sec. 5.2] The text says the keypoint-based DOA correction resulted in a slight decrease in spatial accuracy, but Table 1 shows no change in DOAE for systems (3.1), (4.1), and (5.1); please clarify whether the decrease was observed on a validation set and is therefore not reflected in the table.
  4. [Sec. 5.3] The ensemble rule contains several hand-set parameters (the 20-degree agreement threshold, the requirement of at least two systems, and the Bell/Knock exceptions); please state whether these were tuned on the development set and, if so, treat them as additional selected hyperparameters.
  5. [Reproducibility] The manuscript gives a data link but no code link; a statement on code availability would improve reproducibility, especially for the synthetic data generation and the Cross-Modal Conformer implementation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the reported gains are empirical, and the only self-citation (stpACC) is not load-bearing; the dev-set checkpoint selection is a bias concern, not a circular reduction.

full rationale

The paper's derivation chain is empirical rather than definitional. The stpACC feature is taken from the authors' prior work [19], but it is used as an input feature, not as a fitted parameter; the 10-point RDE improvement is attributed post hoc without an ablation, which is an over-attribution but not a circular step. The Cross-Modal Conformer is an adapted Conformer with cross-attention, not a renamed result, and no equation in the paper reduces a prediction to its inputs by construction. The model-selection protocol in Sec. 5.1 chooses the best checkpoint by F1 on the test partition of the development set; this is a statistical optimism concern that biases the comparison against fixed baselines, but it is not a definitional circularity because the reported F1 is an evaluated score of a trained model rather than a fitted parameter renamed as a prediction. The self-citations ([16], [19], [25]) are not load-bearing: they support architectural choices or feature definitions, and the central claim of outperforming baselines is supported by the reported table, whose main limitation is the selection protocol rather than circular reasoning. Therefore the paper receives a low circularity score, with the minor self-citation noted but not treated as evidence of circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim depends on standard SELD components from cited literature, two domain assumptions about the challenge data (stereo derivation, center-biased framing), and hand-set hyperparameters (loss weight, ensemble threshold, CMC depth). No new physical entities are posited.

free parameters (4)
  • on-screen BCE loss weight = 4.0
    Introduced in Sec. 5.1 to counter on/off-screen imbalance; chosen by hand, not swept.
  • ensemble DOA agreement threshold = 20 degrees
    Sec. 5.3: events require detection by at least two systems with DOAs within 20 degrees; threshold hand-set.
  • CMC depth per stage = 1 layer audio, 2 layers video
    Sec. 2.2: 'we found' single layer sufficed for audio, two for video; no reported sweep.
  • frame sampling rate = 1 fps
    Sec. 2.3: temporal granularity sacrificed for spatial/semantic tokens.
assumptions (3)
  • domain assumption Stereo channels in the dataset are arithmetically derived from FOA and lack inter-channel time/phase differences
    Sec. 3: enables ILD-only spatial features; load-bearing for the feature design.
  • domain assumption Most sound events occur near the center of the frame
    Sec. 2.3: motivates the nonlinear re-framing of OWL-ViT input; if false, the preprocessing distorts relevant content.
  • standard math Multi-ACCDOA and ADPIT loss correctly represent up to N=3 overlapping tracks
    Sec. 2: adopted from [20,21,22,23] without modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videos." pith.science (2026). https://pith.science/paper/XP4GBOIS

@misc{pith2026250704845,
  author       = {Pith},
  title        = {Pith review of: Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XP4GBOIS}},
  note         = {Machine review of arXiv:2507.04845}
}
read the original abstract

This report presents our systems submitted to the audio-only and audio-visual tracks of the DCASE2025 Task 3 Challenge: Stereo Sound Event Localization and Detection (SELD) in Regular Video Content. SELD is a complex task that combines temporal event classification with spatial localization, requiring reasoning across spatial, temporal, and semantic dimensions. The last is arguably the most challenging to model. Traditional SELD architectures rely on multichannel input, which limits their ability to leverage large-scale pre-training due to data constraints. To address this, we enhance standard SELD architectures with semantic information by integrating pre-trained, contrastive language-aligned models: CLAP for audio and OWL-ViT for visual inputs. These embeddings are incorporated into a modified Conformer module tailored for multimodal fusion, which we refer to as the Cross-Modal Conformer. Additionally, we incorporate autocorrelation-based acoustic features to improve distance estimation. We pre-train our models on curated synthetic audio and audio-visual datasets and apply a left-right channel swapping augmentation to further increase the training data. Both our audio-only and audio-visual systems substantially outperform the challenge baselines on the development set, demonstrating the effectiveness of our strategy. Performance is further improved through model ensembling and a visual post-processing step based on human keypoints. Future work will investigate the contribution of each modality and explore architectural variants to further enhance results.

Figures

Figures reproduced from arXiv: 2507.04845 by the authors.

Figure 1
Figure 1. Proposed audio-visual model (left). Blue blocks are [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of synthetic scenes (clockwise from top left) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 43 canonical work pages

  1. [1]

    Alpha” α and “Beta

    INTRODUCTION Sound Event Localization and Detection [1] is a combined task that integrates sound event detection (SED) [2] and sound source local- ization (SSL) [3]. The goal is to identify active sound events from predefined target classes, track their temporal activity, and estimate their spatial positions. SELD systems are crucial for a wide range of r...

    work page arXiv 2019

  2. [2]

    Alpha” ∈ RTα×dk is combined with another modality “Beta

    PROPOSED ARCHITECTURE The proposed model incorporates a primary SELD encoder that ex- tracts SELD embeddings from multichannel input features. These embeddings are then integrated with the CLAP audio embedding through a cross-modal cross-attention mechanism. We adapted the Conformer architecture to accommodate inputs from different modalities. The resulti...

    work page

  3. [3]

    So, we adopted the inter-channel level difference (ILD) as the primary spatial feature for the SELD encoder, alongside log mel spectrograms computed independently from each channel

    ACOUSTIC INPUT FEA TURES Since the left and right audio channels in the dataset are arith- metically derived from FOA signals, rather than captured by two physically separated microphones, they should not present inter- channel time or phase differences. So, we adopted the inter-channel level difference (ILD) as the primary spatial feature for the SELD en...

    work page

  4. [4]

    Knock” and “Bell

    PRE-PROCESSING AND DA TA AUGMENTA TION We pre-trained our model on synthetic data while keeping the CLAP audio encoder and OWL-ViT weights frozen. Synthetic FOA audio was generated using SpatialScaper [28], which con- volves FSD50K sounds [29] with RIRs from various datasets [6, 30, 31, 32, 33, 34, 35]. We created 5,000 one-minute FOA clips averaging 18 e...

    work page

  5. [5]

    off-screen

    EXPERIMENTS 5.1. Implementation Details Audio spectrograms were generated via STFT with a 512-point Hann window and 150-sample hop size. With a sampling rate of 24 kHz, this produces 800 temporal bins for the 5-second input clips. We used 64 mel bins for our spectrograms. For stpACC fea- tures, we applied an STFT with a 1014-point Hann window. This ensure...

    work page 2025

  6. [6]

    Our approach leverages a model that integrates semantically rich feature embeddings from CLAP and OWL-ViT, fused through an adapted Conformer architecture

    CONCLUSION This technical report describes the systems submitted to Task 3 of the DCASE 2025 Challenge. Our approach leverages a model that integrates semantically rich feature embeddings from CLAP and OWL-ViT, fused through an adapted Conformer architecture. The model is pre-trained on large, curated synthetic audio and audio- visual datasets. All submit...

    work page 2025

  7. [7]

    A dataset of dynamic reverberant sound scenes with directional interferers for sound event localization and detection,

    A. Politis et al., “A dataset of dynamic reverberant sound scenes with directional interferers for sound event localization and detection,” in DCASE Workshop, 2021

    work page 2021

  8. [8]

    Sound event localization and detection of over- lapping sources using convolutional recurrent neural networks,

    S. Adavanne et al., “Sound event localization and detection of over- lapping sources using convolutional recurrent neural networks,” IEEE J. of Selected Topics in Sig. Proc., vol. 13, pp. 34–48, 2019

    work page 2019

Show all 50 references
  1. [9]

    Sound event detection using spatial features and convolutional recurrent neural network,

    S. Adavanne et al., “Sound event detection using spatial features and convolutional recurrent neural network,” inICASSP, 2017

    2017

  2. [10]

    Direction of arrival estima- tion for multiple sound sources using convolutional recurrent neural network,

    S. Adavanne, A. Politis, and T. Virtanen, “Direction of arrival estima- tion for multiple sound sources using convolutional recurrent neural network,” in EUSIPCO, 2018, pp. 1462–1466

    2018

  3. [11]

    Audio-visual cross-attention network for robotic speaker tracking,

    X. Qian et al., “Audio-visual cross-attention network for robotic speaker tracking,” IEEE/ACM Trans. Audio, Speech & Lang. Proc. , vol. 31, pp. 550–562, 2023

    2023

  4. [12]

    ForecasterFlexOBM: A multi-view audio-visual dataset for flexible object-based media production,

    D. Berghi et al., “ForecasterFlexOBM: A multi-view audio-visual dataset for flexible object-based media production,” in ICME, 2024

    2024

  5. [13]

    A dataset of reverberant spatial sound scenes with moving sources for sound event localization and detection,

    A. Politis et al., “A dataset of reverberant spatial sound scenes with moving sources for sound event localization and detection,” inDCASE Workshop, 2020

    2020

  6. [14]

    Simple open-vocabulary object detection,

    M. Minderer et al., “Simple open-vocabulary object detection,” in ECCV, 2022

    2022

  7. [15]

    STARSS23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events,

    K. Shimada et al., “STARSS23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events,” in NeurIPS, 2023

    2023

  8. [16]

    Baseline models and evaluation of sound event localization and detection with distance estimation in DCASE 2024 Challenge,

    D. Diaz-Guerra et al., “Baseline models and evaluation of sound event localization and detection with distance estimation in DCASE 2024 Challenge,” in DCASE Workshop, 2024, pp. 41–45

    2024

  9. [17]

    Language models are few-shot learners,

    T. Brown et al., “Language models are few-shot learners,” inAdvances in Neural Information Processing Systems, 2020

    2020

  10. [18]

    Learning transferable visual models from natural language supervision,

    A. Radford et al., “Learning transferable visual models from natural language supervision,” in Int. Conf. on Machine Learning, 2021

    2021

  11. [19]

    AudioGPT: understanding and generating speech, music, sound, and talking head,

    R. Huang et al., “AudioGPT: understanding and generating speech, music, sound, and talking head,” in AAAI Conf. on Artificial Intelli- gence, 2024

    2024

  12. [20]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu et al., “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in ICASSP, 2023

    2023

  13. [21]

    Multi-ACCDOA: Localizing and detecting over- lapping sounds from the same class with auxiliary duplicating permu- tation invariant training,

    K. Shimada et al., “Multi-ACCDOA: Localizing and detecting over- lapping sounds from the same class with auxiliary duplicating permu- tation invariant training,” inICASSP, 2022, pp. 316–320

    2022

  14. [22]

    A four-stage data augmentation approach to resnet- conformer based acoustic modeling for sound event localization and detection,

    Q. Wang et al., “A four-stage data augmentation approach to resnet- conformer based acoustic modeling for sound event localization and detection,” IEEE/ACM Trans. Audio, Speech & Lang. Proc. , vol. 31, pp. 1251–1264, 2023

    2023

  15. [23]

    Fusion of audio and visual embeddings for sound event localization and detection,

    D. Berghi et al., “Fusion of audio and visual embeddings for sound event localization and detection,” in ICASSP, 2024

    2024

  16. [24]

    Resnet-conformer network using multi-scale channel attention for sound event localization and detection in real scenes,

    L. Xue et al., “Resnet-conformer network using multi-scale channel attention for sound event localization and detection in real scenes,” in Int. Conf. on Wireless Comms. & Sig. Proc., 2023

    2023

  17. [25]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati et al., “Conformer: Convolution-augmented transformer for speech recognition,” in Interspeech, 2020, pp. 5036–5040

    2020

  18. [26]

    Reverberation-based features for sound event localization and detection with distance estimation,

    D. Berghi and P. J. B. Jackson, “Reverberation-based features for sound event localization and detection with distance estimation,” ArXiv, vol. abs/2504.08644, 2025

    2025 arXiv

  19. [27]

    Yet, we believe that this alternative sacrifices seman- tic richness, as pooling across channels degrades the learned feature representations

    or this year’s baseline system, apply average pooling across the channel dimension of the ResNet50 output, preserving the 7×7 spatial layout. Yet, we believe that this alternative sacrifices seman- tic richness, as pooling across channels degrades the learned feature represent...

    2025

  20. [28]

    Sound event detection and localization with dis- tance estimation,

    D. A. Krause et al., “Sound event detection and localization with dis- tance estimation,” in EUSIPCO, 2024, pp. 286–290

    2024

  21. [29]

    Event-independent network for polyphonic sound event localization and detection,

    Y . Cao et al., “Event-independent network for polyphonic sound event localization and detection,” in DCASE Workshop, 2020

    2020

  22. [30]

    An improved event-independent network for polyphonic sound event localization and detection,

    Y . Cao et al., “An improved event-independent network for polyphonic sound event localization and detection,” in ICASSP, 2021

    2021

  23. [31]

    Batch normalization: Accelerating deep net- work training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep net- work training by reducing internal covariate shift,” in International Conference on Machine Learning, vol. 37, 2015, pp. 448–456

    2015

  24. [32]

    Leveraging reverberation and visual depth cues for sound event localization and detection with distance estimation,

    D. Berghi and P. J. B. Jackson, “Leveraging reverberation and visual depth cues for sound event localization and detection with distance estimation,” in Techical Report of DCASE Challenge, 2024

    2024

  25. [33]

    Deep residual learning for image recognition,

    K. He et al., “Deep residual learning for image recognition,” in IEEE Conf. on Comp. Vis. & Pattern Recogn. (CVPR), 2016, pp. 770–778

    2016

  26. [34]

    MV ANet: Multi-stage video attention network for sound event localization and detection with source distance estima- tion,

    H. Hong et al., “MV ANet: Multi-stage video attention network for sound event localization and detection with source distance estima- tion,” ArXiv, vol. abs/2411.14153, 2024

    2024 arXiv

  27. [35]

    Spatial Scaper: A library to simulate and aug- ment soundscapes for sound event localization and detection in realis- tic rooms,

    I. R. Roman et al., “Spatial Scaper: A library to simulate and aug- ment soundscapes for sound event localization and detection in realis- tic rooms,” in ICASSP, 2024

    2024

  28. [36]

    FSD50K: an open dataset of human-labeled sound events,

    E. Fonseca et al., “FSD50K: an open dataset of human-labeled sound events,” IEEE/ACM Trans. Audio, Speech & Lang. Proc., vol. 30, pp. 829–852, 2022

    2022

  29. [37]

    METU SPARG Eigenmike em32 Acoustic Impulse Response Dataset v0.1.0,

    O. Olgun and H. Hacihabiboglu, “METU SPARG Eigenmike em32 Acoustic Impulse Response Dataset v0.1.0,” Apr. 2019. [Online]. Available: https://doi.org/10.5281/zenodo.2635758

    2019 doi

  30. [38]

    Dataset of spatial room impulse responses in a variable acoustics room for six degrees-of- freedom rendering and analysis,

    T. McKenzie, L. McCormack, and C. Hold, “Dataset of spatial room impulse responses in a variable acoustics room for six degrees-of- freedom rendering and analysis,” ArXiv, vol. 2111.11882, 2021

    2021 arXiv

  31. [39]

    FMA: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A dataset for music analysis,” ArXiv, vol. abs/1612.01840, 2016

    2016 arXiv

  32. [40]

    A dataset of higher-order ambisonic room impulse responses and 3d models measured in a room with varying furniture,

    G. G ¨otz et al., “A dataset of higher-order ambisonic room impulse responses and 3d models measured in a room with varying furniture,” in Immersive and 3D Audio (I3DA), 2021

    2021

  33. [41]

    Room impulse re- sponse dataset of a recording studio with variable wall paneling mea- sured using a 32-channel spherical microphone array and a B-format microphone array,

    G. Chesworth, A. Bastine, and T. Abhayapala, “Room impulse re- sponse dataset of a recording studio with variable wall paneling mea- sured using a 32-channel spherical microphone array and a B-format microphone array,”Applied Sciences, vol. 14, no. 5, p. 2095, 2024

    2024

  34. [42]

    Data set: Eigenmike-DRIRs, KEMAR 45BA-BRIRs, RIRs and 360° pictures captured at five positions of a small conference room,

    C. Schneiderwind et al., “Data set: Eigenmike-DRIRs, KEMAR 45BA-BRIRs, RIRs and 360° pictures captured at five positions of a small conference room,” in Proc. 45th DAGA, 2019

    2019

  35. [43]

    Generating di- verse audio-visual 360 soundscapes for sound event localization and detection,

    A. S. Roman, A. Chang, G. Meza, and I. R. Roman, “Generating di- verse audio-visual 360 soundscapes for sound event localization and detection,” ArXiv, vol. abs/2504.02988, 2025

    2025 arXiv

  36. [44]

    Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,

    B. A. Plummer et al., “Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,” inICCV, 2015

    2015

  37. [45]

    Robust and adaptive door operation with a mobile robot,

    M. Arduengo et al., “Robust and adaptive door operation with a mobile robot,” Intelligent Service Robotics, p. 409–425, 2021

    2021

  38. [46]

    NitroFusion: High-fidelity single-step diffusion through dynamic adversarial training,

    D.-Y . Chen et al., “NitroFusion: High-fidelity single-step diffusion through dynamic adversarial training,” ArXiv, vol. abs/2412.02030, 2024

    2024 arXiv

  39. [47]

    360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,

    S.-H. Chou et al., “360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,” in IEEE Winter Conf. on Appli- cations of Comp. Vis., 2020

    2020

  40. [48]

    Exploring audio-visual information fusion for sound event localization and detection in low-resource realistic scenarios,

    Y . Jiang et al., “Exploring audio-visual information fusion for sound event localization and detection in low-resource realistic scenarios,” in ICME, 2024, pp. 1–6

    2024

  41. [49]

    Enhanced sound event localization and de- tection in real 360-degree audio-visual soundscapes,

    A. S. Roman et al., “Enhanced sound event localization and de- tection in real 360-degree audio-visual soundscapes,” ArXiv, vol. abs/2401.17129, 2024

    2024 arXiv

  42. [50]

    Ultralytics YOLO,

    G. Jocher, Q. Jing, and A. Chaurasia, “Ultralytics YOLO,” Accessed Jun. 21, 2025, 2023, https://github.com/ultralytics/ultralytics

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.