Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

AV-DTEC: Self-Supervised Audio-Visual Fusion for Drone Trajectory Estimation and Classification

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that AV-DTEC, a self-supervised audio-visual fusion model, estimates drone trajectories with 0.67 m mean error and 99.3% classification accuracy on the MMAUD benchmark, trained without manual annotation.

desk verdict Useful system with open code, but the paper's own ablation shows the proposed fusion modules don't improve trajectory estimation, so the SOTA claim doesn't hold as stated. read the letter →

arxiv 2412.16928 v1 pith:5B2ZSG6D submitted 2024-12-22 cs.SD cs.CVcs.MMeess.AS

Zhenyuan Xiao , Yizhuo Yang , Guili Xu , Xianglong Zeng , Shenghai Yuan This is my paper · ORCID
classification cs.SDcs.CVcs.MMeess.AS
keywords anti-UAVaudio-visualfusionself-supervisedlearningtrajectoryestimationdroneclassificationstate-spacemodelMambaLiDARpseudo-labels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a drone-detection system can be trained without any manual annotation and still beat fully supervised audio-visual baselines at both 3D trajectory estimation and drone-type classification. The system, AV-DTEC, uses LiDAR-generated pseudo-labels as its only supervision, then learns to track and classify drones from audio and video. On the MMAUD real-world dataset it reports a mean trajectory error of 0.67 m and 99.3% classification accuracy while running at 1.98 GFLOPs with 10.16M parameters. If correct, this means an inexpensive camera-plus-microphone setup could replace costly LiDAR-based counter-drone systems after a one-time LiDAR-labelled training phase.

What carries the argument

The load-bearing machinery is the Audio Mamba, a selective state-space model applied separately to the temporal and spectral axes of multi-channel mel-spectrograms. Temporal Mamba scans patches left-to-right along time to capture the time-difference of arrival, and Spectral Mamba scans top-to-bottom to capture spectral attenuation. A learnable token in the sequence carries the aggregated information into a prediction head. The second piece is the feature enhancement module, a residual cross-attention block that fuses auxiliary visual or spectral features into the primary audio feature, and the adaptive adjustment mechanism, a teacher-student setup in which a Faster R-CNN teacher supervises a visual student; the student's predicted drone-existence probability becomes the fusion weight during inference.

What would settle it

One concrete check: on the MMAUD test split, compute the residual distribution between the LiDAR-generated training labels and the survey-grade ground truth. If the mean absolute or signed bias of those residuals approaches or exceeds 0.67 m, AV-DTEC's reported trajectory error cannot be trusted as true localization accuracy; retraining the same architecture on ground-truth labels should not improve APE if the pseudo-labels are adequate.

Watch

Extended reading notes

Core claim

The central discovery is a self-supervised training scheme plus architecture that makes audio the primary modality for drone localization. Rather than asking annotators to label drone positions, the method clusters LiDAR point clouds with DBSCAN to produce pseudo-labels accurate to about one meter; these labels supervise an audio-visual network. The network extracts audio features with a parallel selective state-space model (Audio Mamba) that scans mel-spectrograms along both time and frequency axes, and it extracts visual features with Vision Mamba. A feature enhancement module injects visual features into audio features through residual cross-attention, while a teacher-student adaptive adjustment mechanism learns a scalar that down-weights visual features when the drone is not confidently visible. On the MMAUD benchmark the model reports state-of-the-art results: mean APE 0.67 m and accuracy 99.3%, and it degrades far less than visual or fusion baselines when lighting changes.

Load-bearing premise

The entire training signal rests on LiDAR-generated training labels being accurate to roughly one meter; if those labels carry a larger or biased error, the reported sub-meter trajectory accuracy would not represent true localization quality.

Editorial extensions

If this is right

  • Deployment-time cost drops to a camera and microphone array; LiDAR is needed only to generate training labels, not during inference.
  • Audio-first design keeps performance nearly flat from day to night: the reported mean APE moves from 0.58 m in daylight to 0.75 m at night, while pure-visual baselines collapse to errors above 8 m.
  • Because the model needs no manual annotations, counter-drone systems can be retrained for new environments or new drone types using only unlabelled recordings plus a temporary LiDAR setup.
  • The small footprint (1.98 GFLOPs, 10.16M parameters) puts real-time drone tracking within reach of embedded or mobile devices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported accuracy should be read against the pseudo-label floor: with LiDAR labels accurate to about one meter, a mean APE of 0.67 m suggests the model partially corrects label noise, but the claim is only as strong as the unmeasured bias distribution of those labels.
  • The teacher-student weighting could be tested in reverse: a version that also learns to ignore audio when wind noise dominates might improve robustness beyond the lighting changes tested here.
  • Because the method estimates a 3D trajectory in a hemisphere rather than a full sphere, extending it to 360-degree coverage would require either multiple microphone arrays or a different fusion geometry; the paper does not address that regime.
  • A direct test of the self-supervision hypothesis would be to compare against the same architecture trained on survey-grade ground truth; if performance does not improve, the pseudo-labels are effectively as good as manual annotation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes AV-DTEC, a lightweight audio-visual fusion system for drone trajectory estimation and classification on the MMAUD dataset. The method uses a Mamba/SSM-based audio feature extractor (AMamba with temporal and spectral branches), a Vision Mamba branch, a feature enhancement module based on residual cross-attention, and an adaptive adjustment mechanism driven by a Faster R-CNN teacher. Training uses pseudo-labels generated by an unsupervised LiDAR-based clustering pipeline, avoiding manual annotation. The paper reports mean APE of 0.67 m and classification accuracy of 99.3% on the test set, together with low computational cost (1.98 GFLOPs, 10.16M parameters), and open-sources the code.

Significance. If the reported results are reliable, the paper would make a useful contribution to practical anti-UAV systems: it demonstrates that a relatively small audio-visual model can estimate drone trajectories and classify drone types on real-world multi-modal data without manual annotations, and the choice of audio as the primary modality is well motivated by robustness to lighting changes. The open-source release of code and trained models, the use of a real-world dataset, and the lightweight architecture are clear strengths that increase reproducibility and practical value. However, the central claim of state-of-the-art trajectory estimation is undermined by an internal inconsistency in the ablation study, and the lack of multiple runs or error bars leaves the main quantitative claims unverified.

major comments (4)
  1. [Table IV and Section IV-D] The paper's headline claim that AV-DTEC achieves state-of-the-art trajectory estimation (mean APE 0.67 m in Table I) is contradicted by the paper's own ablation study. In Table IV, the row 'AMamba + Vim + Concate' (no FEM, no AAM) achieves mean APE 0.64 m and light-condition APE 0.52 m, while the full model with FEM and AAM achieves mean APE 0.67 m and light-condition APE 0.58 m. Thus the proposed feature enhancement module and adaptive adjustment mechanism do not improve trajectory estimation over a plain concatenation of the same audio and visual features; on the light condition the full model is worse. The discussion in Section IV-D claims that AAM 'solves the problem of alignment' and that performance is 'less degraded,' but the data do not show that the full model is better than the concatenation baseline on the primary metric. This is load-bearing because the paper explicitly claims state-of-the-art APE. The authors should either provide a statistically significant demonstration that FEM/AAM improve trajectory estimation, or revise the claim that the full model is state-of-the-art in trajectory estimation.
  2. [Section III-D and Fig. 6] The pseudo-label accuracy is the load-bearing premise for the sub-meter trajectory results, but the paper provides only the statement that LiDAR pseudo-labels have 'up to 1-meter accuracy.' Since the reported mean APE is 0.67 m, the model is trained against labels that may be noisier than the errors it reports; without a quantitative analysis of the pseudo-label error distribution (bias, variance, and per-axis errors) and a sensitivity analysis showing that the reported APE is not an artifact of label noise, the trajectory numbers are not fully supported. Figure 6 shows a difference distribution, but no summary statistics or discussion of how this noise affects the L1 training loss in Eq. (5). This concern is distinct from the Table IV inconsistency and needs to be addressed before the trajectory claim can be accepted.
  3. [Section III-B2 and Eqs. (4), (7)] The adaptive adjustment mechanism (AAM) is not specified well enough to be reproduced from the text. Equation (4) states the fusion computation, and Eq. (7) refers to a teacher-student loss Lt-s, but the paper does not define how the binary cross-entropy existence loss and the position loss are combined in Lt-s, how the Faster R-CNN teacher's outputs are converted into the adjustment factor alpha, or how the student model's predictions are used to 'reversely weaken' visual features during inference. Since AAM is one of the two main claimed contributions and the paper argues it is essential for alignment, the absence of these details makes the method under-specified. The code release helps, but the manuscript itself should contain the full formulation.
  4. [All experimental tables] All results are based on a single 7:3 train/test split with no multiple runs, error bars, or significance tests. Differences such as 99.3% vs. 98.9% accuracy, 0.67 m vs. 0.64 m APE, and 99.7% vs. 99.6% light-condition accuracy are small and could easily be within run-to-run variation. The paper should report results over multiple seeds or provide confidence intervals, especially for the ablation comparisons in Tables III and IV that support the claimed benefits of each module.
minor comments (6)
  1. [Title, Abstract, GitHub URL] The system name is inconsistently written: AV-DTEC appears in the title and abstract, but 'AV-DETC' appears in the GitHub URL and in Section IV-D. Please use one consistent spelling.
  2. [Section IV-A] Typo: 'Dateset' should be 'Dataset'.
  3. [Section III] The text says 'teacher-teacher model' in the description of the adaptive adjustment mechanism; this should be 'teacher-student model'.
  4. [Table I caption] The caption says 'Overline: the mean value of day and night,' but it is unclear whether 'Overline' refers to the last two columns or to an overline symbol in the table. Please clarify the notation for the mean columns.
  5. [Section III-D] The pseudo-label generation relies on 'empirically filtering outliers based on point size, shape, and spatial-temporal continuity,' but the exact thresholds are not given, which makes the label generation pipeline hard to reproduce independently.
  6. [Section II-A] The related work section mentions several prior methods but does not clearly state what is missing in them relative to the proposed work, especially with respect to 3D trajectory estimation with audio-visual fusion; a brief comparative table in the text would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the model is trained on external LiDAR pseudo-labels and a pretrained Faster R-CNN teacher, then evaluated against independent survey-graded ground truth, so the reported predictions are not rebuilt from the model's own outputs.

full rationale

The claim chain is self-contained with respect to the model: AV-DTEC is trained with pseudo-labels generated by an unsupervised LiDAR clustering pipeline (Section III-D) and a pretrained Faster R-CNN teacher (Section III-B2), and it is evaluated on the held-out split of MMAUD against survey-graded ground truth (Section IV-A). The LiDAR pseudo-labels are inputs, not predictions of the model, so the reported mean APE of 0.67 m is not equal by construction to any fitted constant; the evaluation is external to the training signal. The closest baseline (AV-FDTI [25]) and the dataset (MMAUD [31]) come from the same research group, but this is a provenance overlap, not a load-bearing self-citation: AV-FDTI is a previously published supervised method and MMAUD provides external ground truth. The paper's own ablation (Table IV) shows that the full FEM+AAM model has worse mean APE (0.67 m) than the concatenation baseline (0.64 m), and Section IV-D2 admits 'Using FEM directly has lower performance than concatenate.' This is a serious correctness and consistency problem for the claimed fusion benefit, but it is not circularity: the numbers are still benchmark results, and no equation reduces the reported trajectory error to the fitting procedure. No step in the derivation chain is defined in terms of its own output, and no prediction is a renamed fit. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The system's performance rests on the standard math of Mamba/SSM, the domain assumption that audio is robust to lighting changes, and the adequacy of LiDAR pseudo-labels as training targets. No new physical entities are introduced; the 'learnable tokens' are network parameters, not new entities.

free parameters (3)
  • Loss balance factors gamma1, gamma2 = gamma1=2, gamma2=0.5
    Set by hand in Section IV-B to balance classification, position, and teacher-student losses; affects the reported accuracy.
  • Patch sizes for audio and image = audio w=4,h=1; image 16x16
    Chosen in Section IV-B; standard hyperparameters but tuned for this task.
  • Pseudo-label outlier filtering thresholds = unspecified
    Section III-D says outliers are filtered 'empirically' based on point size, shape, and continuity, but no thresholds are given; these directly determine label quality.
assumptions (4)
  • standard math The SSM discretization (Eq. 2) correctly models temporal and spectral structures of audio spectrograms.
    The paper applies the Mamba selective state-space model to audio; it assumes the model can capture sound propagation differences.
  • domain assumption Audio features are stable across illumination conditions.
    The design makes audio the primary modality based on this premise; experiments support it, but it is an input to the architecture.
  • domain assumption LiDAR pseudo-labels are accurate to within 1 meter and unbiased.
    Section III-D states the accuracy bound; the method and the reported APE depend on this premise.
  • domain assumption Faster R-CNN pretrained on a generic UAV RGB dataset provides reliable visual existence probabilities on MMAUD.
    The AAM uses this teacher; if it does not transfer, the visual weighting mechanism fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AV-DTEC: Self-Supervised Audio-Visual Fusion for Drone Trajectory Estimation and Classification." pith.science (2026). https://pith.science/paper/5B2ZSG6D

@misc{pith2026241216928,
  author       = {Pith},
  title        = {Pith review of: AV-DTEC: Self-Supervised Audio-Visual Fusion for Drone Trajectory Estimation and Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5B2ZSG6D}},
  note         = {Machine review of arXiv:2412.16928}
}
read the original abstract

The increasing use of compact UAVs has created significant threats to public safety, while traditional drone detection systems are often bulky and costly. To address these challenges, we propose AV-DTEC, a lightweight self-supervised audio-visual fusion-based anti-UAV system. AV-DTEC is trained using self-supervised learning with labels generated by LiDAR, and it simultaneously learns audio and visual features through a parallel selective state-space model. With the learned features, a specially designed plug-and-play primary-auxiliary feature enhancement module integrates visual features into audio features for better robustness in cross-lighting conditions. To reduce reliance on auxiliary features and align modalities, we propose a teacher-student model that adaptively adjusts the weighting of visual features. AV-DTEC demonstrates exceptional accuracy and effectiveness in real-world multi-modality data. The code and trained models are publicly accessible on GitHub \url{https://github.com/AmazingDay1/AV-DETC}.

Figures

Figures reproduced from arXiv: 2412.16928 by the authors.

Figure 1
Figure 1. Our audio-visual fusion model effectively identifies [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. AV-DTEC Architecture. During training, the learnable visual token extracted by Vim is trained through the teacher [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The architecture of TMamba and SMamba block. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The Acc confusion matrix for the classification results of AV-DTEC [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: The difference distribution between pseudo labels [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Test set trajectory estimation: Red curves represent ground truth, blue curves show predicted trajectories. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Autonomous 3D Moving Target Encirclement and Interception with Range measurement

    cs.RO 2025-06 conditional novelty 5.0 of 10

    A two-drone range-only estimator and anti-synchronization controller lets autonomous guardians encircle and intercept a moving hostile UAV in 3D.

  2. TAME: Temporal Audio-based Mamba for Enhanced Drone Trajectory Estimation and Classification

    cs.SD 2024-12 conditional novelty 5.0 of 10

    TAME applies parallel Mamba state-space models to audio spectrograms and reports state-of-the-art drone trajectory estimation and classification on MMAUD, with unresolved evaluation concerns.

  3. Audio Array-Based 3D UAV Trajectory Estimation with LiDAR Pseudo-Labeling

    cs.RO 2024-12 conditional novelty 5.0 of 10

    An audio-only neural network predicts 3D UAV trajectories with 0.48 m average position error on MMAUD, trained with unsupervised LiDAR trajectory estimates as pseudo-labels.

  4. Unsupervised UAV 3D Trajectories Estimation with Sparse Point Clouds

    cs.CV 2024-12 conditional novelty 4.0 of 10

    An unsupervised LiDAR clustering and spline method estimates UAV 3D trajectories from sparse point clouds, reporting 1.35 m RMSE on the MMAUD v2/v3 benchmark.

Reference graph

Works this paper leans on

32 extracted references · 19 canonical work pages · cited by 4 Pith papers

  1. [1]

    Unmanned aerial systems for civil applications: A review,

    H. Gonz ´alez-Jorge, J. Mart ´ınez-S´anchez, M. Bueno, and P. Arias, “Unmanned aerial systems for civil applications: A review,” Drones, vol. 1, no. 1, p. 2, 2017

    work page 2017

  2. [2]

    Drone-surveillance for search and rescue in natural disaster,

    B. Mishra, D. Garg, P. Narang, and V . Mishra, “Drone-surveillance for search and rescue in natural disaster,” Computer Communications, vol. 156, pp. 1–10, 2020

    2020

  3. [3]

    Mobile technology platform project for monitoring and border surveillance of illegal migration and smuggling of goods,

    J. Jev ˇc´ak, L. Choma, P. Petr ´ıˇcek, H. N ´emethov´a, S. Mak ´o, M. Pil ´at, T. Straka, and V . Polishchuk, “Mobile technology platform project for monitoring and border surveillance of illegal migration and smuggling of goods,” in 2019 Modern Safety Technologies in Transportation (MOSATT). IEEE, 2019, pp. 71–75

    work page 2019

  4. [4]

    Air-to-air visual detection of micro-uavs: An experimental evaluation of deep learning,

    Y . Zheng, Z. Chen, D. Lv, Z. Li, Z. Lan, and S. Zhao, “Air-to-air visual detection of micro-uavs: An experimental evaluation of deep learning,” IEEE Robotics and automation letters , vol. 6, no. 2, pp. 1020–1027, 2021

    2021

  5. [5]

    Dronechase: A mobile and automated cross-modality system for continuous drone tracking,

    N. V ora, Y . Wu, J. Liu, and P. Nguyen, “Dronechase: A mobile and automated cross-modality system for continuous drone tracking,” in Proceedings of the Ninth Workshop on Micro Aerial V ehicle Networks, Systems, and Applications , 2023, pp. 8–14

    work page 2023

  6. [6]

    Multi-modal uav detection, classification and tracking algorithm–technical report for cvpr 2024 ug2 challenge,

    T. Deng, Y . Zhou, W. Wu, M. Li, J. Huang, S. Liu, Y . Song, H. Zuo, Y . Wang, Y . Yue et al. , “Multi-modal uav detection, classification and tracking algorithm–technical report for cvpr 2024 ug2 challenge,” arXiv preprint arXiv:2405.16464 , 2024

    arXiv 2024

  7. [7]

    Av-pedaware: Self- supervised audio-visual fusion for dynamic pedestrian awareness,

    Y . Yang, S. Yuan, M. Cao, J. Yang, and L. Xie, “Av-pedaware: Self- supervised audio-visual fusion for dynamic pedestrian awareness,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 1871–1877

    2023

  8. [8]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    arXiv 2023

Show all 32 references
  1. [9]

    Vision-based anti-uav de- tection and tracking,

    J. Zhao, J. Zhang, D. Li, and D. Wang, “Vision-based anti-uav de- tection and tracking,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 25 323–25 334, 2022

    2022

  2. [10]

    Investigation of uav detection in images with complex backgrounds and rainy artifacts,

    A. Munir, A. J. Siddiqui, and S. Anwar, “Investigation of uav detection in images with complex backgrounds and rainy artifacts,” in Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 221–230

    2024

  3. [11]

    Anti-uav: A large multi-modal benchmark for uav tracking,

    N. Jiang, K. Wang, X. Peng, X. Yu, Q. Wang, J. Xing, G. Li, J. Zhao, G. Guo, and Z. Han, “Anti-uav: A large multi-modal benchmark for uav tracking,” arXiv preprint arXiv:2101.08466 , 2021

    2021 arXiv

  4. [12]

    The 2nd anti-uav workshop & challenge: Methods and results,

    J. Zhao, G. Wang, J. Li, L. Jin, N. Fan, M. Wang, X. Wang, T. Yong, Y . Deng, Y . Guo et al. , “The 2nd anti-uav workshop & challenge: Methods and results,” arXiv preprint arXiv:2108.09909 , 2021

    2021 arXiv

  5. [13]

    The 3rd anti-uav workshop & challenge: Methods and results,

    J. Zhao, J. Li, L. Jin, J. Chu, Z. Zhang, J. Wang, J. Xia, K. Wang, Y . Liu, S. Gulshad et al. , “The 3rd anti-uav workshop & challenge: Methods and results,” arXiv preprint arXiv:2305.07290 , 2023

    2023 arXiv

  6. [14]

    A real-time and lightweight method for tiny airborne object detection,

    Y . Lyu, Z. Liu, H. Li, D. Guo, and Y . Fu, “A real-time and lightweight method for tiny airborne object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3016–3025

    2023

  7. [15]

    Anti- uav410: A thermal infrared benchmark and customized scheme for tracking drones in the wild,

    B. Huang, J. Li, J. Chen, G. Wang, J. Zhao, and T. Xu, “Anti- uav410: A thermal infrared benchmark and customized scheme for tracking drones in the wild,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

    2023

  8. [16]

    A large- scale uav audio dataset and audio-based uav classification using cnn,

    Y . Wang, Z. Chu, I. Ku, E. C. Smith, and E. T. Matson, “A large- scale uav audio dataset and audio-based uav classification using cnn,” in 2022 Sixth IEEE International Conference on Robotic Computing (IRC). IEEE, 2022, pp. 186–189

    2022

  9. [17]

    Exploiting wavelet scattering transform & 1d-cnn for unmanned aerial vehicle detection,

    M. Ali and K. Nathwani, “Exploiting wavelet scattering transform & 1d-cnn for unmanned aerial vehicle detection,” IEEE Signal Processing Letters, 2024

    2024

  10. [18]

    A vehicle-mounted radar- vision system for precisely positioning clustering uavs,

    G. Wu, F. Zhou, K. K. Wong, and X.-Y . Li, “A vehicle-mounted radar- vision system for precisely positioning clustering uavs,” IEEE Journal on Selected Areas in Communications , 2024

    2024

  11. [19]

    On onboard lidar-based flying object detection,

    M. Vrba, V . Walter, V . Pritzl, M. Pliska, T. B ´aˇca, V . Spurn`y, D. He ˇrt, and M. Saska, “On onboard lidar-based flying object detection,” IEEE Transactions on Robotics , 2024

    2024

  12. [20]

    Soundspaces: Audio-visual navigation in 3d environments,

    C. Chen, U. Jain, C. Schissler, S. V . A. Gari, Z. Al-Halah, V . K. Ithapu, P. Robinson, and K. Grauman, “Soundspaces: Audio-visual navigation in 3d environments,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI

    2020

  13. [21]

    Springer, 2020, pp. 17–36

    2020

  14. [22]

    Is someone speaking? exploring long-term temporal features for audio- visual active speaker detection,

    R. Tao, Z. Pan, R. K. Das, X. Qian, M. Z. Shou, and H. Li, “Is someone speaking? exploring long-term temporal features for audio- visual active speaker detection,” in Proceedings of the 29th ACM international conference on multimedia , 2021, pp. 3927–3935

    2021

  15. [23]

    Egocentric deep multi- channel audio-visual active speaker localization,

    H. Jiang, C. Murdock, and V . K. Ithapu, “Egocentric deep multi- channel audio-visual active speaker localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 10 544–10 552

    2022

  16. [24]

    There is more than meets the eye: Self-supervised multi-object detection and tracking with sound by distilling multimodal knowledge,

    F. R. Valverde, J. V . Hurtado, and A. Valada, “There is more than meets the eye: Self-supervised multi-object detection and tracking with sound by distilling multimodal knowledge,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp...

    2021

  17. [25]

    Self-supervised moving vehicle tracking with stereo sound,

    C. Gan, H. Zhao, P. Chen, D. Cox, and A. Torralba, “Self-supervised moving vehicle tracking with stereo sound,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 7053–7062

    2019

  18. [26]

    Av-fdti: Audio-visual fusion for drone threat identification,

    Y . Yang, S. Yuan, J. Yang, T. H. Nguyen, M. Cao, T.-M. Nguyen, H. Wang, and L. Xie, “Av-fdti: Audio-visual fusion for drone threat identification,” Journal of Automation and Intelligence , 2024

    2024

  19. [27]

    An image is worth 16x16 words: Transformers for image recogni- tion at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recogni- tion at scale,” International Conference on Learning Representations (ICLR), 2021

    2021

  20. [28]

    Vision mamba: Efficient visual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417 , 2024

    2024 arXiv

  21. [29]

    Faster r-cnn: Towards real-time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems , vol. 28, 2015

    2015

  22. [30]

    Yolov4: Op- timal speed and accuracy of object detection,

    A. Bochkovskiy, C.-Y . Wang, and H.-Y . M. Liao, “Yolov4: Op- timal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020

    2004 arXiv

  23. [31]

    Clustering- based learning for uav tracking and pose estimation,

    J. Xiao, P. Pisutsin, C. W. Tsao, and M. Feroskhan, “Clustering- based learning for uav tracking and pose estimation,” arXiv preprint arXiv:2405.16867, 2024

    2024 arXiv

  24. [32]

    MMAUD: A Comprehensive Multi-Modal Anti- UA V Dataset for Modern Miniature Drone Threats,

    S. Yuan, Y . Yang, T. H. Nguyen, T.-M. Nguyen, J. Yang, F. Liu, J. Li, H. Wang, and L. Xie, “MMAUD: A Comprehensive Multi-Modal Anti- UA V Dataset for Modern Miniature Drone Threats,” in Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA). I...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.