Pith. sign in

REVIEW 4 major objections 6 minor 54 references

MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Cross-modal knowledge distillation improves when each training sample is routed to its own best teacher, and a mask module retunes that teacher to the student's behavior.

desk verdict The mixture-of-teachers routing is a fresh engineering idea, but the MaskNet adaptation step makes teachers imitate the student, so the paper's cross-modal transfer claim is not supported and the empirical gains lack statistical backing. read the letter →

arxiv 2507.07015 v1 pith:OPLMJCEW submitted 2025-07-09 cs.CV cs.LGcs.MM

Hui Li , Pengfei Yang , Juanyang Chen , Le Dong , Yanxin Chen , Quan Wang This is my paper · ORCID
classification cs.CVcs.LGcs.MM
keywords cross-modalknowledgedistillationmixtureofteachersdynamicroutingMaskNetdriftmultimodallearningteacher-studenttraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that cross-modal knowledge distillation fails for two fixable reasons: the teacher is chosen once and statically, and the teacher's inductive bias drifts from the student's so its guidance is partly unusable. The proposed answer is MST-Distill, a three-stage framework in which a pool of multimodal and cross-modal teachers is first trained together, each teacher is then specialized toward the target student by a learnable MaskNet that re-filters its internal features, and finally a per-sample routing network lets the student dynamically pick the top teacher for each input. The authors argue this matters because real deployments often run a single-modality student (for example a vision-only edge model) that could benefit from knowledge in other modalities without paying the cost of full multimodal inference. They report that on five multimodal datasets, spanning classification and semantic segmentation, this adaptive pipeline consistently beats single-teacher and static-teacher distillation baselines.

What carries the argument

The load-bearing machinery is the mixture of specialized teachers: a pool of $N$ teachers formed by inserting independent MaskNet modules into selected layers of the multimodal and auxiliary unimodal models, plus a GateNet router that scores all $N$ teachers from the student's logits and selects the top $k$. MaskNet itself is a multi-head self-attention block followed by a linear layer and sigmoid, producing a soft mask that is multiplied onto intermediate teacher features; it is trained alone, with all base-model parameters frozen, to minimize the KL divergence between the specialized teacher's softened output and the student's output. The argument leans on two identities being separable: that the routing network can learn which teacher transfers best for a given sample, and that MaskNet can remove only the teacher's modality-specific discrepancies while preserving its complementary cross-modal knowledge.

What would settle it

Run a control on a dataset with a strong teacher advantage (for example the visual student on RAVDESS): replace MaskNet with direct fine-tuning of the frozen teacher's logits toward the student under the same KL loss and compute budget, then compare transfer gains; if the two gains are statistically indistinguishable, the adapted teacher is a noisy copy of the student and the reported improvement is self-training rather than knowledge transfer.

Watch

Extended reading notes

Core claim

The paper's central claim is that no fixed teacher, whether multimodal or cross-modal, is reliably the best source of supervision for a target-modality student: transferability is asymmetric across modality directions, varies per dataset, and even varies per instance. MST-Distill therefore replaces the static teacher with a mixture of specialized teachers and an instance-level router. In the first stage, all modality-specific models are trained jointly with task loss and bidirectional KL alignment, without gradient detachment, to create aligned starting points. In the second stage, independently parameterized MaskNet modules are inserted at chosen layers of each frozen teacher and trained only against the student's softened outputs, with the stated purpose of suppressing modality-specific discrepancies and reconstructing teacher representations that match the student's behavior. In the third stage, the student's own logits are fed to a small routing network whose top-$k$ teacher selection (by default $k=1$) determines which specialized teacher supervises each sample, with a load-balancing loss keeping the full pool in use. The paper reports that this design outperforms existing response-, feature-, and relation-based distillation methods as well as recent cross-modal methods on AV-MNIST, RAVDESS, VGGSound-50k, CrisisMMD-V2, and NYU-Depth-V2.

Load-bearing premise

The claim stands on the premise that MaskNet, trained only to make the teacher's outputs match the student's, suppresses modality-specific discrepancies while leaving the teacher's complementary cross-modal knowledge intact; the training signal itself cannot distinguish those two things.

Editorial extensions

If this is right

  • A student trained under MST-Distill should be able to absorb cross-modal knowledge even when the static choice of a single teacher would fail, because the router can pick a multimodal teacher for some samples and a cross-modal teacher for others.
  • The stage ordering matters: the paper's ablations show that dynamic distillation applied directly to static teachers yields minimal gains, implying that collaborative initialization and MaskNet specialization are prerequisites for the router to be useful.
  • The framework transfers beyond classification to dense prediction, since the same three stages with pixel-level distillation improve both RGB and depth students on NYU-Depth-V2.
  • With all teachers selected at once ($k=4$), routing degrades into uniform averaging and performance drops, so the per-sample selection mechanism itself, not mere teacher diversity, is credited with the gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the routing mechanism is not tied to modality; the same GateNet-plus-MaskNet recipe could be tested on a pool of same-modality teachers with different architectures, where per-sample selection might absorb model-capacity or initialization differences instead of modality differences.
  • Editorial extension: routing probabilities over training (reported for RAVDESS) could be compared against an oracle that, for each validation sample, picks the teacher whose distillation gave the largest student improvement; the gap would quantify how much headroom remains in the router.
  • Editorial extension: the paper leaves implicit that MaskNet's per-sample reconstruction could act as an instance-level curriculum, gradually steering easy samples toward one teacher and hard samples toward another; a testable prediction is that routing entropy should be higher on hard samples early in training and should decrease over time.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MST-Distill, a three-stage cross-modal knowledge distillation framework. In the first stage, all modality-specific models (one per modality plus a multimodal model) are jointly trained with task and bidirectional KL-alignment losses. In the second stage, MaskNet modules are inserted into the frozen teacher models and trained individually to minimize the KL divergence between each teacher's output and the frozen target-modality student's output. In the third stage, a GateNet routes each sample to the top-k adapted teachers, and the student is trained with a weighted combination of task loss, KL-divergence to the selected teachers' softened outputs, and a load-balancing loss. Experiments are reported on four multimodal classification datasets and on NYU-Depth-V2 for semantic segmentation, together with ablations, routing analyses, and hyperparameter studies. The paper claims that MST-Distill significantly outperforms existing cross-modal knowledge distillation methods.

Significance. If correctly established, the idea of a mixture of specialized teachers with instance-level routing would be a useful contribution to cross-modal knowledge distillation, and the paper has several strengths: it clearly motivates the path-selection and knowledge-drift problems with empirical analyses in Appendix A, it releases code, and it evaluates across five datasets spanning vision, audio, text, and RGB-depth. However, the central mechanism described in Stages 2 and 3 is circular as written: the teachers are adapted to match the frozen student, and the same adapted teacher outputs are then used as distillation targets for that student. In addition, the empirical support for the headline claim is weak because only mean accuracies are reported and several improvements over the strongest baselines are tiny or negative. As a result, the paper does not currently establish that complementary cross-modal knowledge is transferred.

major comments (4)
  1. [§3.2, Eq. (8); §3.3, Eq. (14)] The Stage-2 objective is a pure output-distribution match: ℓ_j = KL(Q^j_{m_t} ∥ Q^j_{m_δ(j)}) minimizes the divergence between each specialized teacher and the frozen target student. There is no term that preserves the teacher's original knowledge, no reconstruction loss for the masked features defined in Eq. (6), and no decomposition of modality-specific discrepancies from complementary cross-modal information. Stage 3 then distills these student-shaped teachers back into the same student via Eq. (14), so the supervision is a function of the student's own outputs. The reported gains are therefore equally explained by self-distillation or regularization, and the paper's central claim that complementary cross-modal knowledge is transferred is not supported.
  2. [Tables 1 and 2] The abstract's claim of 'significantly outperforms existing state-of-the-art' is not verifiable from the reported statistics. Only mean accuracies over five runs are given, with no standard deviations, confidence intervals, or significance tests. Several reported differences against the strongest baseline DML are below 0.5% absolute or negative, for example AV-MNIST Audio (MST-Distill 0.4381 vs DML 0.4393), RAVDESS Audio (0.7174 vs 0.7202), VGGSound Visual (0.4595 vs 0.4601), CrisisMMD Image (0.5495 vs MLLD MM 0.5549), and NYU-Depth-V2 RGB OA (0.5396 vs DML 0.5455). The claimed superiority is therefore not established.
  3. [Table 3, ablation study] The ablation study compares pipeline variants but does not include a self-distillation control in which the frozen student itself, or a teacher exactly matching the student, is used as the distillation target. Such a control is necessary to determine whether the improvements in settings (d), (e), and (f) come from cross-modal knowledge transfer or from the regularization effect of matching the student's own outputs. Without this control, the contribution of MaskNet to cross-modal transfer remains unidentified.
  4. [§3.1, Eq. (4)] The collaborative initialization stage jointly trains all teachers and the student with a bidirectional KL loss and no gradient detachment. This means the 'teachers' are already optimized partly to match the target student's output distribution before the MaskNet adaptation begins. This co-adaptation further confounds the attribution of later gains to complementary cross-modal knowledge, and it should be discussed or controlled for.
minor comments (6)
  1. [§3.2, Eq. (6)] The text repeatedly states that MaskNet 'reconstructs teacher representations,' but no reconstruction loss is defined; Eq. (6) only produces a masked feature. Please either add a reconstruction objective or rephrase the claims throughout the paper.
  2. [§3.2, Eq. (9)] The notation Q^j_{m_t} is used for the student output in Eq. (8), but Eq. (9) defines Q^j_{m_i} only for teacher outputs f^j_{m_i}; the student's distribution should be defined separately to avoid ambiguity.
  3. [Figure 2] Some labels in Figure 2 read 'MsakNetT' and appear to be typos for 'MaskNet'; please correct them and ensure the stage names in the figure match the section names.
  4. [Table 2] The text says MST-Distill shows 'superior performance across all evaluation metrics' and 'ranking first in five of six metrics and second in the remaining one,' but the RGB OA row is below DML; please rephrase to 'best or second-best' for consistency.
  5. [References] Reference [4] contains a typo ('Wjournali Xie'); please proofread the reference list and check all entries for accuracy.
  6. [Appendix C.1, Algorithm 1] In line 15, the specialized teachers are loaded with Stage-1 base parameters and Stage-2 MaskNet parameters, while the student is loaded from Stage 1; please clarify whether the student is re-initialized or fine-tuned at the start of Stage 3.

Circularity Check

2 steps flagged · score 8.0 of 10

Stage-2 MaskNet adaptation fits each specialized teacher to the frozen student's logits (Eq. 8), so the Stage-3 distillation target (Eq. 14) reduces to the student's own output; the claimed cross-modal transfer is self-distillation.

  1. fitted input called prediction [Section 3.2, Eqs. (8)–(10)]
    "Subsequently, all model parameters except those of MaskNet are frozen, and each MaskNet is trained independently to align the behavior of its corresponding specialized teacher with that of the target student, guided by response consistency. ... ℓ_j = KL(Q^j_mt ∥ Q^j_mδ(j)) ... L^j_S2 = 1/B Σ_{b=1}^B ℓ^{(b)}_j."

    MaskNet parameters are the only trainable parameters, and the objective is a pure KL match between the specialized teacher's output and the frozen target student's output. The paper claims MaskNet suppresses 'modality-specific discrepancies' and 'reconstructs teacher representations,' but Eq. (10) contains no reconstruction term, no teacher-fidelity term, and no decomposition separating discrepancies from complementary knowledge. By construction, any teacher output that disagrees with the student is penalized; the complementary cross-modal signal that motivated the mixture is exactly the kind of information that disagrees, so it is not constrained to survive. The 'specialized teacher' is therefore a fitted function of the student's own output, not an independently informative teacher.

  2. self definitional [Section 3.3, Eq. (14)]
    "ℓ_DKD = Σ_{j∈Ttop-k} KL(Q^j_mδ(j) ∥ P_mt), where Q^j_mδ(j) denotes the softened output distribution from the j-th specialized teacher of modality mδ(j), and P_mt is the student's output distribution."

    Stage 2 minimized KL(Q^j_mt ∥ Q^j_mδ(j)) with Q^j_mt the frozen student output, so at the start of Stage 3 the specialized teacher distribution Q^j_mδ(j) approximately equals the student's own output Q^j_mt. Equation (14) then reduces to KL(Q^j_mt ∥ P_mt), a self-distillation or output-regularization loss on the same student. The cross-modal 'knowledge' being distilled is thus defined as a function of the student itself. The paper's central claim—that complementary cross-modal knowledge is transferred—is not supported by the equations; the reported improvements are equally consistent with self-distillation or logit smoothing. The Table 3 ablation compares pipeline variants but does not measure what information survives MaskNet adaptation, so it does not resolve this reduction.

full rationale

The circularity is internal to the method, not a matter of self-citation. Stage 2 trains MaskNet to make each specialized teacher's output distribution match the frozen target student's output distribution (Eq. 8/10), and Stage 3 then uses those same student-shaped teacher outputs as distillation targets for the same student (Eq. 14). By the paper's own equations, the teacher signal is a fitted function of the student's logits; no objective preserves teacher information that disagrees with the student. Consequently, the 'mixture of specialized teachers' supervision is approximately self-distillation, and the claim that the framework transfers complementary cross-modal knowledge is not independently demonstrated. The empirical comparisons against baselines and the component ablations are useful evidence that the overall training recipe helps, but they do not establish that the mechanism is cross-modal knowledge transfer rather than regularization. The paper does not import a uniqueness theorem or rely on load-bearing self-citation, so the circularity here is structural rather than bibliographic.

Assumptions & free parameters 5 free parameters · 3 assumptions · 2 invented entities

The framework rests on several hand-chosen hyperparameters (weights, decays, top-k, temperature) and on the unvalidated assumption that MaskNet alignment preserves complementary information. No free parameters are fitted to the final test numbers in a statistical sense, but the choice of k=1 and the mask layer selections were guided by the same benchmarks used for evaluation.

free parameters (5)
  • lambda_1 = 1, halved every 30 epochs
    Weight for distillation loss in Eq. 18; chosen by hand, decays over training.
  • lambda_2 = 1, reduced 10% every 10 epochs
    Weight for load-balancing loss; chosen by hand.
  • top-k = 1
    Number of teachers selected per instance; set to 1 after sensitivity analysis (Fig. 6).
  • temperature tau = not specified in text
    Softmax temperature in Eq. 3; not reported.
  • number of MaskNet layers N_mi = varies, intermediate+penultimate
    Number of specialized teachers per base teacher; chosen by hand.
assumptions (3)
  • domain assumption KL divergence between softened outputs is an appropriate measure of teacher-student knowledge alignment
    Used in Eqs. 2, 8, 14; standard in KD.
  • domain assumption Training teachers and students jointly (Stage 1) creates a shared representation beneficial for later transfer
    Stage 1 objective (Eq. 4) with bidirectional KL; not proven to be necessary or sufficient.
  • ad hoc to paper MaskNet can selectively suppress modality-specific discrepancies while preserving complementary cross-modal information
    Core assumption of Stage 2; contradicted by the fact that the loss aligns teacher to student, which removes information that differs from the student.
invented entities (2)
  • MaskNet
    purpose: Learnable soft-mask module inserted into teacher layers to reconstruct teacher representations and align teacher output with student output
    Only evidence is the paper's own ablations; no external validation that it preserves complementary knowledge.
  • GateNet
    purpose: Instance-level routing network that selects top-k specialized teachers for each sample
    Described only within this paper; no independent evaluation of routing quality beyond aggregate accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillation." pith.science (2026). https://pith.science/paper/OPLMJCEW

@misc{pith2026250707015,
  author       = {Pith},
  title        = {Pith review of: MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OPLMJCEW}},
  note         = {Machine review of arXiv:2507.07015}
}
read the original abstract

Knowledge distillation as an efficient knowledge transfer technique, has achieved remarkable success in unimodal scenarios. However, in cross-modal settings, conventional distillation methods encounter significant challenges due to data and statistical heterogeneities, failing to leverage the complementary prior knowledge embedded in cross-modal teacher models. This paper empirically reveals two critical issues in existing approaches: distillation path selection and knowledge drift. To address these limitations, we propose MST-Distill, a novel cross-modal knowledge distillation framework featuring a mixture of specialized teachers. Our approach employs a diverse ensemble of teacher models across both cross-modal and multimodal configurations, integrated with an instance-level routing network that facilitates adaptive and dynamic distillation. This architecture effectively transcends the constraints of traditional methods that rely on monotonous and static teacher models. Additionally, we introduce a plug-in masking module, independently trained to suppress modality-specific discrepancies and reconstruct teacher representations, thereby mitigating knowledge drift and enhancing transfer effectiveness. Extensive experiments across five diverse multimodal datasets, spanning visual, audio, and text, demonstrate that our method significantly outperforms existing state-of-the-art knowledge distillation methods in cross-modal distillation tasks. The source code is available at https://github.com/Gray-OREO/MST-Distill.

Figures

Figures reproduced from arXiv: 2507.07015 by the authors.

Figure 1
Figure 1. Illustration of two key challenges in cross-modal [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the MST-Distill framework in a two-modality setting, consisting of three stages: Collaborative Initialization [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overall architecture of MaskNet. A soft mask is [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Average routing probabilities of specialized teachers [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Performance trends under different top-𝑘 values in dynamic knowledge distillation. Student performance across modalities is distinguished by color and marker. Effect of Top-𝑘 Teacher Selection. Our sensitivity analysis exam￾ines parameter 𝑘, which controls the number o…
Figure 5
Figure 5. Figure 5: Box plots of OA improvements under different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Performance comparison of multimodal and uni [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 9
Figure 9. Figure 9: Grad-CAM visualizations comparing the visual attention of a multimodal teacher and a unimodal visual student on [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: t-SNE visualizations of teacher features before and after MaskNet processing on AV-MNIST dataset. (a) Early phase [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Grad-CAM visualizations of different models on four CrisisMMD-V2 samples. Each row shows the visualizations [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 52 canonical work pages

  1. [1]

    Firoj Alam, Ferda Ofli, and Muhammad Imran. 2018. CrisisMMD: Multimodal Twitter Datasets from Natural Disasters. In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 12

    work page 2018

  2. [2]

    Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2018. Multi- modal Machine Learning: A Survey and Taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 2 (2018), 423–443

    work page 2018

  3. [3]

    Lluis Castrejon, Yusuf Aytar, Carl Vondrick, Hamed Pirsiavash, and Antonio Tor- ralba. 2016. Learning Aligned Cross-Modal Representations from Weakly Aligned Data. In 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2940–2949

    work page 2016

  4. [4]

    Honglie Chen, Wjournali Xie, Andrea Vedaldi, and Andrew Zisserman. 2020. VGGSound: A Large-Scale Audio-Visual Dataset. InIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 721–725

    work page 2020

  5. [5]

    Jun-Ho Choi and Jong-Seok Lee. 2019. EmbraceNet: A Robust Deep Learning Architecture for Multimodal Classification.Information Fusion 51 (2019), 259–270

    work page 2019

  6. [6]

    Inseop Chung, SeongUk Park, Jangho Kim, and Nojun Kwak. 2020. Feature-Map- Level Online Adversarial Knowledge Distillation. In International Conference on Machine Learning. 2006–2015

    work page 2020

  7. [7]

    Xinyi Ding, Tao Han, Yili Fang, and Eric Larson. 2023. An Approach for Combin- ing Multimodal Fusion and Neural Architecture Search Applied to Knowledge Tracing. Applied Intelligence 53, 9 (2023), 11092–11103

    work page 2023

  8. [8]

    Hao Dong, Ismail Nejjar, Han Sun, Eleni Chatzi, and Olga Fink. 2023. SimMMDG: A Simple and Effective Framework for Multi-Modal Domain Generalization. Advances in Neural Information Processing Systems 36 (2023), 78674–78695

    work page 2023

Show all 54 references
  1. [9]

    Yunfeng Fan, Wenchao Xu, Haozhao Wang, Junxiao Wang, and Song Guo. 2023. PMR: Prototypical Modal Rebalance for Multimodal Learning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 20029–20038

    2023

  2. [10]

    Ryan Khushan Ghamandi, Ravi Kiran Kattoju, Yahya Hmaiti, Mykola Maslych, Eugene Matthew Taranta, Ryan P McMahan, and Joseph LaViola. 2024. Unlocking Understanding: An Investigation of Multimodal Communication in Virtual Reality Collaboration. In Proceedings of the 2024 CHI Con...

    2024

  3. [11]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. In Proceedings of International Conference on Learning Representations

    2024

  4. [12]

    Zhiwei Hao, Jianyuan Guo, Kai Han, Yehui Tang, Han Hu, Yunhe Wang, and Chang Xu. 2023. One-for-All: Bridge the Gap between Heterogeneous Archi- tectures in Knowledge Distillation. Advances in Neural Information Processing Systems 36 (2023), 79570–79582

    2023

  5. [13]

    Caner Hazirbas, Lingni Ma, Csaba Domokos, and Daniel Cremers. 2016. FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Archi- tecture. In Asian Conference on Computer Vision . Springer, 213–228

    2016

  6. [14]

    Xiao He, Chang Tang, Xin Zou, and Wei Zhang. 2023. Multispectral Object Detection via Cross-Modal Conflict-Aware Learning. In Proceedings of the 31st ACM International Conference on Multimedia . 1465–1474

    2023

  7. [15]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531

    2015 arXiv

  8. [16]

    Weipeng Hu, Bohong Liu, Haitang Zeng, Yanke Hou, and Haifeng Hu. 2022. Adversarial Decoupling and Modality-Invariant Representation Learning for Visible-Infrared Person Re-Identification. IEEE Transactions on Circuits and Systems for Video Technology 32, 8 (2022), 5095–5109

    2022

  9. [17]

    Fushuo Huo, Wenchao Xu, Jingcai Guo, Haozhao Wang, and Song Guo. 2024. C2KD: Bridging the Modality Gap for Cross-Modal Knowledge Distillation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 16006–16015

    2024

  10. [18]

    Ying Jin, Jiaqi Wang, and Dahua Lin. 2023. Multi-level logit distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 24276–24285

    2023

  11. [19]

    Donghwa Kim and Pilsung Kang. 2022. Cross-Modal Distillation with Audio-Text Fusion for Fine-Grained Emotion Classification Using BERT and Wav2vec 2.0. Neurocomputing 506 (2022), 168–183

    2022

  12. [20]

    Michael Kleinman, Alessandro Achille, and Stefano Soatto. 2023. Critical Learn- ing Periods for Multisensory Integration in Deep Networks. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 24296–24305

    2023

  13. [21]

    Ke Li, Fuyu Dong, Di Wang, Shaofeng Li, Quan Wang, Xinbo Gao, and Tat-Seng Chua. 2024. Show Me What and Where Has Changed? Question Answering and Grounding for Remote Sensing Change Detection. arXiv:2410.23828

    2024 arXiv

  14. [22]

    Ke Li, Di Wang, Haojie Xu, Haodi Zhong, and Cong Wang. 2024. Language- Guided Progressive Attention for Visual Grounding in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing (2024)

    2024

  15. [23]

    Zhihui Li, Pengfei Xu, Xiaojun Chang, Luyao Yang, Yuanyuan Zhang, Lina Yao, and Xiaojiang Chen. 2023. When Object Detection Meets Knowledge Distillation: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 8 (2023), 10555–10579

    2023

  16. [24]

    Xiao Liang, Yanlei Zhang, Di Wang, Haodi Zhong, Ronghan Li, and Quan Wang

  17. [25]

    Yupeng Liang, Ryosuke Wakaki, Shohei Nobuhara, and Ko Nishino. 2022. Multi- modal Material Segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 19800–19808

    2022

  18. [26]

    Steven R Livingstone and Frank A Russo. 2018. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A Dynamic, Multimodal Set of Facial and Vocal Expressions in North American English. PloS one 13, 5 (2018), e0196391

    2018

  19. [27]

    Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. 2023. Are Multimodal Transformers Robust to Missing Modality?. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 18177–18186

    2023

  20. [28]

    Wenxuan Ma, Shuang Li, Lincan Cai, and Jingxuan Kang. 2024. Learning Modality Knowledge Alignment for Cross-Modality Transfer. In Proceedings of the 41st International Conference on Machine Learning . 33777–33793

    2024

  21. [29]

    Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, Andrew Y Ng, et al. 2011. Multimodal Deep Learning. InInternational Conference on Machine Learning, Vol. 11. 689–696

    2011

  22. [30]

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. 2019. Relational Knowl- edge Distillation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3967–3976

    2019

  23. [31]

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2024. TimeChat: A Time-Sensitive Multimodal Large Language Model for Long Video Understanding. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 14313–14323

    2024

  24. [32]

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for Thin Deep Nets. In Proceedings of International Conference on Learning Representations

    2015

  25. [33]

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In Proceedings of the IEEE Inter- national Conference on Computer Vision . 618–626

    2017

  26. [34]

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. 2012. Indoor Segmentation and Support Inference from RGBD Images. In European Conference on Computer Vision. Springer, 746–760

    2012

  27. [35]

    Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew G Wilson. 2021. Does Knowledge Distillation Really Work?. In Advances in Neural Information Processing Systems , Vol. 34. 6906–6919

    2021

  28. [36]

    Jun Sun, Shoukang Han, Yu-Ping Ruan, Xiaoning Zhang, Shu-Kai Zheng, Yulong Liu, Yuxin Huang, and Taihao Li. 2023. Layer-Wise Fusion with Modality Inde- pendence Modeling for Multi-Modal Emotion Recognition. In Annual Meeting of the Association for Computational Linguistics . 658–670

    2023

  29. [37]

    Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive Representation Distillation. In Proceedings of International Conference on Learning Representa- tions

    2020

  30. [38]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. Advances in Neural Information Processing Systems 30 (2017)

    2017

  31. [39]

    Valentin Vielzeuf, Alexis Lechervy, Stéphane Pateux, and Frédéric Jurie. 2018. CentralNet: A Multilayer Approach for Multimodal Fusion. In European Confer- ence on Computer Vision Workshops

    2018

  32. [40]

    Di Wang, Caiping Zhang, Quan Wang, Yumin Tian, Lihuo He, and Lin Zhao. 2022. Hierarchical Semantic Structure Preserving Hashing for Cross-Modal Retrieval. IEEE Transactions on Multimedia 25 (2022), 1217–1229

    2022

  33. [41]

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al . 2024. InternVideo2: Scaling Foundation Models for Multimodal Video Understanding. InEuropean Conference on Computer Vision. Springer, 396–416

    2024

  34. [42]

    Yake Wei and Di Hu. 2024. MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance. In International Conference on Machine Learning . PMLR, 52559–52572

    2024

  35. [43]

    Nan Wu, Stanislaw Jastrzebski, Kyunghyun Cho, and Krzysztof J Geras. 2022. Characterizing and Overcoming the Greedy Nature of Learning in Multi-Modal Deep Neural Networks. In International Conference on Machine Learning . PMLR, 24043–24055

    2022

  36. [44]

    Zihui Xue, Zhengqi Gao, Sucheng Ren, and Hang Zhao. 2023. The Modality Fo- cusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation. In Proceedings of International Conference on Learning Representations

    2023

  37. [45]

    Zihui Xue, Sucheng Ren, Zhengqi Gao, and Hang Zhao. 2021. Multimodal Knowl- edge Expansion. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 854–863

    2021

  38. [46]

    Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont- Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023. Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning. In 2023 IEEE/CVF Conference on Computer Vision and Pat...

    2023

  39. [47]

    Jiangchao Yao, Shengyu Zhang, Yang Yao, Feng Wang, Jianxin Ma, Jianwei Zhang, Yunfei Chu, Luo Ji, Kunyang Jia, Tao Shen, et al . 2022. Edge-Cloud Polariza- tion and Collaboration: A Comprehensive Survey for AI. IEEE Transactions on Knowledge and Data Engineering 35, 7 (2022), ...

    2022

  40. [48]

    Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. 2017. A Gift from Knowl- edge Distillation: Fast Optimization, Network Minimization and Transfer Learn- ing. In 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4133–4141

    2017

  41. [49]

    Li Zhang and Xiangqian Wu. 2022. Latent Space Semantic Supervision Based on Knowledge Distillation for Cross-Modal Retrieval. IEEE Transactions on Image Processing 31 (2022), 7154–7164

    2022

  42. [50]

    Yedi Zhang, Peter Latham, et al. 2024. Understanding Unimodal Bias in Multi- modal Deep Linear Networks. In International Conference on Machine Learning , Vol. 235. PMLR

    2024

  43. [51]

    Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. 2018. Deep Mutual Learning. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4320–4328

    2018

  44. [52]

    Jing Zhao, Xijiong Xie, Xin Xu, and Shiliang Sun. 2017. Multi-View Learning Overview: Recent Progress and New Challenges. Information Fusion 38 (2017), 43–54

    2017

  45. [53]

    Jinxing Zhou, Dan Guo, and Meng Wang. 2022. Contrastive Positive Sample Propagation along the Audio-Visual Event Line. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 6 (2022), 7239–7257. MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge...

    2022

  46. [2024]

    In Proceedings of the 32nd ACM International Conference on Multimedia

    Divide and Conquer: Isolating Normal-Abnormal Attributes in Knowledge Graph-Enhanced Radiology Report Generation. In Proceedings of the 32nd ACM International Conference on Multimedia . 4967–4975

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.