Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LINK claims that adaptively balancing audio and visual contributions and injecting pseudo-label semantics suppresses cross-modal noise, raising audio event F1 on the LLP dataset to 63.4 from 61.2.

desk verdict A plausible empirical extension of VALOR++ on LLP, but the main 'adaptive' loss as written is mathematically vacuous—the trainable μ collapses to 1—so the ablation results imply the published formula is not what was implemented. read the letter →

arxiv 2412.20872 v2 pith:QQBZZDL2 submitted 2024-12-30 cs.CV

Langyu Wang , Bingke Zhu , Yingying Chen , Jinqiao Wang This is my paper · ORCID
classification cs.CV
keywords audio-visualvideoparsingweakly-supervisedlearningmodalitymisalignmentadaptiveinteractionpseudo-labelsCLIPCLAPtemporal-spatialattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LINK, a weakly-supervised audio-visual video parsing method built on the observation that audio and visual events in real videos are often misaligned, so naively fusing the two modalities injects noise. LINK claims to fix this by learning to balance modality contributions: a temporal-spatial attention module and adaptively weighted cross-modal interaction decide how much each modality should trust the other per segment, and a cosine-similarity-based loss reweights training samples so that boundary-like, mildly misaligned segments dominate optimization. A third module converts CLIP/CLAP pseudo-labels into text-encoded semantic features and fuses them into each modality's features as prior knowledge, suppressing the other modality's noise. On the LLP benchmark, LINK reports the highest audio-segment F1 (69.7) and audio-event F1 (63.4) among compared methods, and improves audio-visual event parsing over the VALOR++ baseline. If these results hold, adaptive interaction weighting is a more direct response to modality misalignment than stronger fusion alone.

What carries the argument

The method has three load-bearing components. The temporal-spatial attention module (TSAM) applies channel and spatial attention over segment features and then passes them through a cross-modal interaction module with four learnable weights ($\alpha_1, \alpha_2, \beta_1, \beta_2$) that scale how much one modality's attended features are added to the other. The segmented audio-visual semantic similarity loss (S-LOSS) reweights the audio-visual similarity term $\lambda L_{\text{avss}}$ with a piecewise weight $\lambda$ based on the cosine similarity of segment features, boosting the $(-0.2, 0)$ range and down-weighting clearly similar or clearly dissimilar pairs. The pseudo-label semantic interaction module (PLSIM) reads event categories from uni-modal pseudo-labels, converts them to text prompts such as 'A photo of …' and 'this is a sound of …', encodes these with frozen CLIP/CLAP text encoders, and fuses the resulting semantic features into the audio and visual branches via multiplicative scaling and additive bias. Each module targets the same failure mode—misaligned modalities contaminating prediction—from a different angle.

What would settle it

Measure the correlation between the cosine similarity used in Eq. 11 and human-annotated audio-visual alignment on the LLP validation set (where segment-level labels exist), then retrain LINK with the reweighting function replaced by a constant weight; if the correlation is not positive and removing the weighting does not hurt audio F1, the central mechanism is unsupported.

Watch

Extended reading notes

Core claim

The central claim is that, for weakly-supervised audio-visual video parsing, the harmful effect of non-aligned audio and visual events can be reduced by making every stage of interaction aware of how well the two modalities agree. LINK does this in three places: it reweights the audio-visual feature similarity loss with a piecewise function of the cosine similarity between CLIP and CLAP segment features, giving extra weight to the $(-0.2, 0)$ range where interaction is ambiguous rather than clearly helpful or harmful; it lets four trainable parameters control the strength of each cross-modal attention path; and it injects semantic features derived from uni-modal pseudo-labels into each modality's final representation. The paper reports that on the LLP dataset this combination raises audio segment-level F1 from 68.1 to 69.7 and audio event-level F1 from 61.2 to 63.4 compared with VALOR++, with smaller gains on visual and joint audio-visual metrics.

Load-bearing premise

The method assumes that the cosine similarity between the CLIP audio feature and the CLAP visual feature of a segment reliably indicates whether cross-modal interaction helps that segment's prediction; if the similarity does not track true event alignment, the reweighting in Eq. 11 will amplify the wrong samples.

Editorial extensions

If this is right

  • If the reweighting truly isolates useful boundary samples, similar piecewise loss schedules could be applied to other weakly-supervised multimodal tasks, such as audio-visual event localization or sound source separation.
  • The pseudo-label semantic interaction module implies that frozen text encoders can act as a denoising prior for uni-modal branches; this could be extended to other label spaces beyond the 25 LLP event classes.
  • Since the gains are largest on audio uni-modal events and smaller on audio-visual joint events, the method suggests that misalignment chiefly hurts modality-specific predictions, and that joint-event prediction may need a different remedy.
  • The four learnable interaction weights provide a per-model estimate of how much each modality should trust the other, which could be analyzed as a diagnostic of dataset alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the cosine-similarity gating in Eq. 11 is a form of curriculum over alignment; one could test it against an oracle gating that uses the true segment-level labels to see how much of the gain comes from the particular choice of the $(-0.2, 0)$ range.
  • Inference: the approach suggests that CLIP/CLAP text encoders can serve as a shared semantic space for de-noising both modalities; a natural test is using the same module with different backbone encoders to see if the benefit transfers.
  • Inference: the conclusion notes that multi-modal event prediction is left for future work, which implies the adaptive-interaction gains are concentrated in uni-modal branches, so the method's next test is whether joint-event accuracy can be improved by the same mechanisms.
  • Inference: because the improvements over VALOR++ are concentrated in audio metrics, the method implicitly challenges the common practice of symmetric fusion and predicts that asymmetric fusion will generalize better on other misaligned multimodal benchmarks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes LINK, a weakly-supervised audio-visual video parsing method evaluated on the LLP benchmark. It combines three components: TSAM, which applies CBAM-style temporal-spatial attention and an AV-Adapter-inspired cross-modal interaction with four trainable weights; S-LOSS, which reweights the audio-visual segment-similarity loss by the cosine similarity of audio and visual features; and PLSIM, which fuses CLIP/CLAP text-encoded pseudo-label semantics into uni-modal features. Experiments report F1 improvements over VALOR++ and other recent baselines on all reported metrics, with the largest gains in audio uni-modal event prediction. The main claim is that adaptively balancing modality contributions and injecting pseudo-label semantics suppress noise from non-aligned audio-visual events.

Significance. If the results are reproducible, the paper provides a concise, component-level recipe for improving audio-visual video parsing over the current state of the art. The strengths are the direct comparison with VALOR++, the component-wise ablation study, and the consistent audio-side gains. However, the significance is limited by the small margins on the joint audio-visual metrics (0.2 F1) and by a technical problem in the S-LOSS formulation, which as written degenerates under training. The pseudo-label semantics also come from a teacher trained on the same weak labels, so the 'a priori' claim is weaker than stated. These issues are fixable, but they must be addressed before the empirical claims can be taken at face value.

major comments (4)
  1. [§II-C, Eq. (11)] The weighting λ in Eq. (11) collapses to the unweighted loss during training, so the S-LOSS component cannot explain the reported gains. Since e^{|1−μ|} ≥ 1 for every real μ, the definition gives λ ≥ 1 for all s, with the global minimum λ = 1 reached at μ = 1; for s ∈ (−0.2,0) one has λ = e^{|1−μ|}, and for s ∈ [0,1] one has λ = e^{|1−μ|}(1−s)+s ≥ 1. Because μ appears only in λ and λ multiplies the nonnegative L_avss in Eq. (12), unconstrained gradient-based minimization of the total loss drives μ toward 1, reducing S-LOSS to the plain L_avss already present in the VALOR++ baseline. The ablation row 'w/ S-LOSS' in Table II (e.g., audio event F1 61.2 → 62.7) cannot be produced by Eq. (11) as written; please correct the formula or disclose the actual implementation (e.g., a frozen or constrained μ, a particular initialization, or a stop-gradient), and report the learned value of μ if it is trainable.
  2. [§III-B, Table I] No variance or significance information is reported. The improvements over VALOR++ on the joint audio-visual metrics are only 0.2 F1 at both segment level (62.1 vs. 61.9) and event level (55.7 vs. 55.5). With a single run, these margins are statistically uninterpretable. Please report mean and standard deviation over at least three seeds and, ideally, a paired test or exact p-values for the main comparisons and the ablation rows.
  3. [§II-D, §III-A] The pseudo-labels used by PLSIM come from VALOR++ [8], whose teacher was trained on the same LLP weak labels, so the semantic priors are not independent of the benchmark. This does not by itself invalidate the comparison to VALOR++, which also consumes pseudo-labels, but it weakens the paper's claim that PLSIM injects 'a priori knowledge' from language. Please discuss this dependence and, if feasible, include a sensitivity experiment with pseudo-labels from a teacher trained on a disjoint source or with corrupted pseudo-labels.
  4. [§III-C, Table II] The ablation labeled 'w/ TSAM(T)' bundles temporal-spatial attention with the four adaptive cross-modal interaction weights (α1, α2, β1, β2) of CMIM, so it does not isolate the mechanism that supposedly balances modality contributions. Please ablate the CMIM weights separately (for example, by fixing them to 1 or 0) and report the learned values of α1, α2, β1, β2; without this, the reader cannot tell whether the model actually learns an adaptive balance or simply benefits from the attention module.
minor comments (7)
  1. [§II-B, Eq. (3)] Equation (3) uses f^a_t in the second factor of the definition of \hat f^m_t for both m=a and m=v; this appears to be a typo for f^m_t, and the two attention branches should be written symmetrically.
  2. [§II-B, Eqs. (4)–(5)] In Eqs. (4) and (5), the softmax arguments are written as \hat f^a_t \hat f^v_t without a transpose or a specified dimension; please clarify whether the softmax is over the temporal dimension or the feature dimension.
  3. [§II-C, Eq. (10)] The definition of r as the intersection over union of audio and visual events should state whether it is computed per segment or per event category; the alignment between s and r in the MSE needs to be explicit.
  4. [§II-C, Eq. (8)] The pseudo-label notation \hat y^{mgt}_t is unclear; it is later replaced by \hat y^m_t in §II-D, so please standardize the superscripts.
  5. [§III-A] The experimental setup does not report the optimizer, learning rate, batch size, number of epochs, or initialization of μ and the four CMIM weights; these details are necessary for reproducibility, and a code-release statement would help.
  6. [References] Reference [13] is cited with the year 2019, but the CLAP paper (Wu et al.) was published in 2023; please verify and update the bibliographic entry.
  7. [§II-B, text after Eq. (2)] The module is called 'temporal attention' in Eq. (1) but 'channel attention' in the paragraph after Eq. (2); please use consistent terminology.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; main risk is the degenerate S-LOSS trainable weight, which is a correctness/reproducibility issue rather than circularity.

full rationale

The paper's final claims are F-scores on the fully annotated LLP test set, so no fitted parameter is renamed as a prediction. The adaptive weights alpha, beta, and mu are trained on the weakly labeled training split and evaluated on held-out labels; that is standard supervised training, not circularity. The S-LOSS weight in Eq. (11), lambda = e^{|1-mu|} on (-0.2,0) and lambda = e^{|1-mu|} + (1-e^{|1-mu|})s on [0,1], satisfies lambda >= 1 for all s, with pointwise minimum lambda = 1 at mu = 1. Because Eq. (12) adds lambda * L_avss with nonnegative MSE loss, minimizing with respect to mu drives mu toward 1, making the reweighting vacuous. This is a genuine mathematical degeneracy and possible formula/implementation mismatch, but it is not a circularity: no output quantity is defined in terms of the target prediction, and the reported ablation gains cannot be explained by the published loss. The pseudo-labels used for supervision and in PLSIM come from a fixed VALOR teacher trained on the same LLP weak labels; this is self-training/distillation and reduces the independence of the semantic priors, but the test evaluation does not use ground-truth labels to generate them, so the benchmark result is not forced by construction. There is no load-bearing uniqueness theorem, no answer smuggled through a self-citation, and no renaming of a known empirical pattern as a new derivation. The dominant dependency on the VALOR++ line is a baseline choice, not a circular step. Score 0 reflects that the paper's derivation chain is not circular, while noting the S-LOSS degeneracy as a separate correctness risk.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three domain assumptions: the reliability of pseudo-labels, the validity of cosine similarity as an alignment proxy, and the transferability of text-encoded event semantics. The only hand-set numerical choices are the -0.2 threshold and the trainable interaction and loss weights, all fit on the LLP training set.

free parameters (4)
  • mu (trainable exponent/offset in S-LOSS weighting) = not reported
    Eq (11) introduces a trainable parameter mu to control the weight lambda for cosine similarity ranges; value is learned on the LLP training set.
  • alpha1, alpha2, beta1, beta2 (CMIM fusion weights) = not reported
    Eqs (4)-(5) use four learnable scalars to gate cross-modal interaction; these are fitted during training and are central to the adaptive interaction claim.
  • cosine similarity threshold -0.2 = -0.2
    Manual hyperparameter in Eq (11) that splits the reweighting ranges; chosen by hand, not derived.
  • MLP parameters in PLSIM (Delta1 to Delta4) = not reported
    Eqs (14)-(15) use four MLPs to map pseudo-label text embeddings into scale and bias parameters; their weights are learned.
assumptions (4)
  • domain assumption Pseudo-labels from VALOR teacher are sufficiently accurate to serve as semantic priors and as targets for L_label and Lavss.
    Sec II-D combines pseudo labels with video-level labels and treats them as ground truth for auxiliary losses; if the teacher is systematically wrong, the model inherits its errors.
  • domain assumption Cosine similarity between audio and visual CLIP/CLAP features is a faithful measure of modality alignment and event correspondence.
    Eq (11) reweights the auxiliary loss based solely on this cosine similarity; if similarity does not track alignment, the weighting emphasizes wrong samples.
  • domain assumption Natural-language templates ('A photo of', 'this is a sound of') transfer event categories to CLIP/CLAP text space semantically.
    Sec II-D constructs text features from pseudo-label event names and uses them to modulate visual and audio features; this assumes the text encoder embeds those phrases close to relevant visual and audio features.
  • domain assumption Standard training on the LLP training split transfers to the LLP validation and test splits.
    The benchmark protocol assumes the held-out annotations measure generalization; this is standard but unstated in detail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing." pith.science (2026). https://pith.science/paper/QQBZZDL2

@misc{pith2026241220872,
  author       = {Pith},
  title        = {Pith review of: LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QQBZZDL2}},
  note         = {Machine review of arXiv:2412.20872}
}
read the original abstract

Audio-visual video parsing focuses on classifying videos through weak labels while identifying events as either visible, audible, or both, alongside their respective temporal boundaries. Many methods ignore that different modalities often lack alignment, thereby introducing extra noise during modal interaction. In this work, we introduce a Learning Interaction method for Non-aligned Knowledge (LINK), designed to equilibrate the contributions of distinct modalities by dynamically adjusting their input during event prediction. Additionally, we leverage the semantic information of pseudo-labels as a priori knowledge to mitigate noise from other modalities. Our experimental findings demonstrate that our model outperforms existing methods on the LLP dataset.

Figures

Figures reproduced from arXiv: 2412.20872 by the authors.

Figure 1
Figure 1. Modality non-aligned samples from LLP. Existing method is vulner [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of LINK. We use temporal-spatial attention and cross modal interaction module to enhance the expression of feature, and merge the semantic information from pseudo label with uni-modal feature. The pseudo labels are extracted by VALOR[8]. II. PROPOSED METHOD A. Problem Statement The AVVP task aims to identify the event of every seg￾ment into audio event, visual event and audio-visual event, together wit… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsing

    cs.MM 2025-05 conditional novelty 5.0 of 10

    TeMTG fuses CLIP/CLAP text embeddings with audio and visual features and applies K-hop graph attention to achieve state of the art segment-level event parsing on the LLP dataset.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [8]

    Modality-independent teachers meet weakly-supervised audio-visual event parser

    Y . Lai, Y . Chen, Y .Wang, “Modality-independent teachers meet weakly-supervised audio-visual event parser.” in Advances in Neural Information Processing Systems, 2023, vol. 36, pp. 73633-73651

    work page 2023

  2. [1]

    Audio- visual event localization in unconstrained videos,

    Y . Tian, J. Shi, B. Li, Z. Duan, and C. Xu, “Audio- visual event localization in unconstrained videos,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 247–263

    work page 2018

  3. [2]

    Dual attention matching for audio-visual event localization,

    Y . Wu, L. Zhu, Y . Yan, and Y . Yang, “Dual attention matching for audio-visual event localization,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 6292–6300

    work page 2019

  4. [3]

    Learning to answer questions in dynamic audio-visual scenarios,

    G. Li, Y Wei, Y . Tian, C. Xu, Ji-Rong Wen, and Di Hu. “Learning to answer questions in dynamic audio-visual scenarios,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 19108-19118

    work page 2022

  5. [4]

    Listen to look: Action recognition by previewing audio,

    R. Gao, T. Oh, K. Grauman, and L. Torresani, “Listen to look: Action recognition by previewing audio,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10457-10467

    work page 2020

  6. [5]

    Unified multisensory per- ception: Weakly-supervised audio-visual video parsing,

    Y . Tian, D. Li, and C. Xu, “Unified multisensory per- ception: Weakly-supervised audio-visual video parsing,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, August 23–28, 2020. Springer, 2020, pp. 436–454

    work page 2020

  7. [6]

    MM-Pyramid: Multimodal pyramid attentional network for audio-visual event localization and video parsing,

    J. Yu, Y . Cheng, R.-W. Zhao, R. Feng, and Y . Zhang, “MM-Pyramid: Multimodal pyramid attentional network for audio-visual event localization and video parsing,” in Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 6241–6249

    work page 2022

  8. [7]

    Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Tasks

    H. Duan, Y . Xia, M. Zhou, L. Tang, J. Zhu, and Z. Zhao, “Cross-modal prompts: Adapting large pretrained mod- els for audio-visual downstream tasks,” arXiv preprint arXiv:2311.05152, 2023

    work page Pith review arXiv 2023

Show all 25 references
  1. [9]

    Label-anticipated Event Disentanglement for Audio-Visual Video Parsing,

    J. Zhou, D. Guo, Y . Mao, Y . Zhong, X.n Chang, and M. Wang, “Label-anticipated Event Disentanglement for Audio-Visual Video Parsing,” in Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), 2024, pp. 1–22

    2024

  2. [10]

    Cbam: Convo- lutional block attention module,

    S. Woo, J. Park, J. Lee, and I. Kweon, “Cbam: Convo- lutional block attention module,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3-19

    2018

  3. [11]

    Towards Efficient Audio-Visual Learners via Empowering Pre-trained Vi- sion Transformers with Cross-Modal Adaptation,

    K. Wang, Y , Tian, D. Hatzinakos, “Towards Efficient Audio-Visual Learners via Empowering Pre-trained Vi- sion Transformers with Cross-Modal Adaptation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1837-1846

    2024

  4. [12]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. Kim, C. Hallacy, A. Ramesh, G. Goh, et al, “Learning transferable visual models from natural language supervision,” in International conference on machine learning, 2021, pp. 8748-8763

    2021

  5. [13]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 201...

    2019

  6. [14]

    Balanced multimodal learning via on-the-fly gradient modulation,

    X. Peng ,Y .Wei, A.Deng, D. Wang, and D. Hu, “Balanced multimodal learning via on-the-fly gradient modulation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 8238-8247

    2022

  7. [15]

    What makes training multi-modal classification networks hard?

    W.Wang, D. Tran, M.Feiszli, “What makes training multi-modal classification networks hard?” in Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 12695-12705

    2020

  8. [16]

    Scal- ing multimodal pre-training via cross-modality gradient harmonization,

    J. Wu, Y .Liang , H, Akbari , Z. Wang, C. Yu, “Scal- ing multimodal pre-training via cross-modality gradient harmonization,” in Advances in Neural Information Pro- cessing Systems, 2022, vol. 35, pp. 36161-36173

    2022

  9. [17]

    Text-IF: Leveraging Semantic Text Guidance for Degradation- Aware and Interactive Image Fusion,

    X. Yi , H. Xu ,H. Zhang , L. Tang, J. Ma, “Text-IF: Leveraging Semantic Text Guidance for Degradation- Aware and Interactive Image Fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27026-27035

    2024

  10. [18]

    Language- driven All-in-one Adverse Weather Removal,

    H. Yang , L. Pan, Y . Yang , and W. Liang, “Language- driven All-in-one Adverse Weather Removal,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 24902-24912

    2024

  11. [19]

    Multi-modal grouping network for weakly-supervised audio-visual video parsing,

    S. Mo, Y . Tian, “Multi-modal grouping network for weakly-supervised audio-visual video parsing,” in Ad- vances in Neural Information Processing Systems, 2022, vol.35, pp. 34722-34733

    2022

  12. [20]

    Joint-modal label denoising for weakly-supervised audio-visual video parsing,

    H. Cheng, Z. Liu, H. Zhou, C. Qian, W. Wu, L. Wang, “Joint-modal label denoising for weakly-supervised audio-visual video parsing,” in Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), 2022, pp. 431–448

    2022

  13. [21]

    Collecting cross-modal presence-absence evidence for weakly-supervised audio- visual event perception,

    J. Gao, M. Chen, C. Xu, “Collecting cross-modal presence-absence evidence for weakly-supervised audio- visual event perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp, 18827–18836

    2023

  14. [22]

    CoLeaF: A Contrastive-Collaborative Learning Frame- work for Weakly Supervised Audio-Visual Video Pars- ing,

    F. Sardari, A. Mustafa, P. Jackson, and A. Hilton, “CoLeaF: A Contrastive-Collaborative Learning Frame- work for Weakly Supervised Audio-Visual Video Pars- ing,” arXiv preprint arXiv:2405.10690, 2024

    2024 arXiv

  15. [23]

    CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsing,

    Y . Chen , R.Guo ,X, Liu , et al, “CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsing,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 8421-8425

    2024

  16. [24]

    V ALOR: Vision- Audio-Language Omni-Perception Pretraining Model and Dataset,

    J. Liu, S. Chen, X. He, et al. “V ALOR: Vision- Audio-Language Omni-Perception Pretraining Model and Dataset,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  17. [25]

    Multi-grained representa- tion learning for cross-modal retrieval,

    S. Zhao, L. Xu, Y . Liu, et al. “Multi-grained representa- tion learning for cross-modal retrieval,” in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp, 2194-2198

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.