Pith. sign in

REVIEW 3 major objections 6 minor 63 references

Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalities

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes HARDY-MER, a training framework that improves multimodal emotion recognition when audio, text, or video inputs are missing by estimating per-sample difficulty and retrieving similar support examples for hard cases, report

desk verdict A sensible integration of hardness-aware curriculum and retrieval, but the main robustness claim is undermined by a train-time self-retrieval leak. read the letter →

arxiv 2508.06800 v2 pith:OMTTSFUS submitted 2025-08-09 cs.LG cs.AI

Rui Liu , Haolin Zuo , Zheng Lian , Hongyu Yuan , Qi Fan This is my paper · ORCID
classification cs.LGcs.AI
keywords multimodalemotionrecognitionmissingmodalitiescurriculumlearninghardsampleminingretrieval-augmentedtrainingmutualinformationreconstructionerrorIEMOCAP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a training method called HARDY-MER for multimodal emotion recognition when some input modalities (audio, text, video) are missing at inference. The central idea is to score how hard each training sample is to learn—by how well available modalities reconstruct the missing one and how much information modalities share—then to give harder samples more help during training by retrieving and appending similar labeled examples. The authors report that this consistently outperforms existing missing-modality methods on IEMOCAP and CMU-MOSEI, with average weighted-accuracy gains of roughly 4.4 points, 3.0 points, and 1.4 points over the best competing method in three settings. The method does not need retrieval at test time; the trained model predicts from whatever modalities are present.

What carries the argument

The multi-view hardness score h = (1 + exp(-$\beta$ * (alpha1 * h_dir + alpha2 * h_ind)))^-1, combining direct reconstruction error and indirect mutual information; and the hardness-aware retrieval scheduler k' = ceil(h*k), which determines how many similar samples are appended to each training input.

What would settle it

Train HARDY-MER on a variant where the retrieved support samples are replaced by random samples of the same emotion class (keeping k' the same); if average accuracy does not drop, the hardness-aware retrieval content is not responsible for the reported gains. Alternatively, swap the training retrieval database for one built from a different dataset; if gains vanish, the method does not generalize to unseen distributions.

Watch

Extended reading notes

Core claim

HARDY-MER claims that sample-specific training hardness, estimated from reconstruction error and cross-modal mutual information, can be turned into a dynamic curriculum that improves emotion recognition under missing modalities. For each input, the framework computes a unified hardness score in (0,1), retrieves the most semantically similar samples from per-modality feature banks, and keeps only k' = ceil(h*k) of them, so harder samples get more support. These support samples are concatenated with the input during training, and the model is trained with classification and reconstruction losses. The authors report new state-of-the-art average results under six missing-modality settings, with

Load-bearing premise

During training the model sees retrieved similar samples appended to its input, but at inference retrieval is switched off, so the claimed gains depend on the model learning transferable unimodal and multimodal features rather than exploiting the retrieved context as a shortcut.

Editorial extensions

If this is right

  • If the claim holds, training-time retrieval augmentation is enough to improve missing-modality accuracy without changing the inference model or requiring external data at test time.
  • Per-condition metrics improve across all six missing-modality settings on IEMOCAP and on most settings on CMU-MOSEI; average gains are statistically significant (p<0.05) by the paper's T-test.
  • The largest gains occur under video-only input, suggesting the method helps most when the remaining modality is least informative.
  • Removing either hardness component or fixing k' degrades performance, implying the adaptive curriculum is the active ingredient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the model learns to use retrieved samples as a crutch rather than as support; an editor's inference is that the strongest validation would compare against retrieving random or same-label samples with the same k' schedule.
  • A natural extension is to apply the same hardness-aware retrieval curriculum to other incomplete-input tasks such as multimodal sentiment analysis with noisy or corrupted modalities, or to audiovisual speech recognition.
  • The mutual information approximation via cross-attention is a practical choice; a stricter estimator might change hardness rankings, so results could be sensitive to that approximation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes HARDY-MER, a training-time framework for multimodal emotion recognition with missing modalities. It first estimates per-sample hardness using reconstruction errors ("direct hardness") and cross-modal mutual information ("indirect hardness"), then uses retrieval from modality-specific FAISS indexes to augment each training sample with a hardness-dependent number of support samples, with more supports for harder samples. The final model is trained on the original incomplete input plus the retrieved supports; at inference no retrieval is used. Experiments on IEMOCAP (4-class and 6-class) and CMU-MOSEI report improved weighted accuracy/accuracy over baselines across six missing-modality settings, with a claimed new state of the art.

Significance. If the results are valid, the combination of hardness-aware curriculum learning with retrieval augmentation is a novel and practically interesting idea for missing-modality MER, and the promised code release would facilitate reproducibility. The paper includes ablations for each hardness component, the adaptive curriculum, retrieval features, and index construction choices. However, two load-bearing issues currently undermine the central claim: (1) the retrieval database is not protected against self-retrieval, so the training signal can leak the missing modality directly into the input, and (2) the signature of the indirect-hardness term contradicts the paper's own definition of hardness. Both issues require re-running experiments after fixing the protocol.

major comments (3)
  1. [§3.3.2–3.3.3] The feature database is built from training samples (Step 2.1), and Step 2.2 queries it with the same sample's available-modality embeddings without excluding the query itself. Since Eq. (18) gives k' = ceil(h·k) with h in (0,1) and k=5, at least one retrieved sample is always used, and the query's own all-modality record is the nearest neighbor. The support set therefore supplies exactly the modality that is marked missing at training time, while §3.3.3 states inference uses no retrieval. The model can minimize reconstruction/classification loss by copying the missing modality from the retrieved self rather than by learning to predict from available modalities. This is a train/test distribution mismatch that directly affects the central missing-modality claim. The w/o retrieval features ablation removes all support and cannot isolate this leakage. The authors should exclude self from th
  2. [§2.1, Eq. (9), Eq. (13)] The motivation states that weak cross-modal consistency makes samples hard, but Eq. (9) defines indirect hardness as the sum of positive mutual information values, so higher MI yields higher h_ind, i.e., stronger consistency is scored as harder. Eq. (13) then minimizes -h_ind, which maximizes MI—consistent with treating high MI as desirable, not hard. This sign contradiction means the hardness score is not a faithful implementation of the paper's own hardness concept, and the dynamic curriculum may emphasize exactly the samples the authors intend to down-weight. The definition of h_ind or the motivation needs to be changed, and the hardness-adaptive component must be re-evaluated.
  3. [Table 1] The table claims significance with '*' and 'T-test' but no standard deviations, confidence intervals, or number of runs are reported anywhere. A paired t-test on the Average column requires multiple independent runs; with a single run per condition, the p-value cannot be computed. Please report mean±std over at least 5 random seeds and full test details. Without this, the 'consistently outperforms' claim is not statistically supported.
minor comments (6)
  1. [Abstract / Footnote 1] The code URL is inconsistent: the abstract gives https://github.com/HARDY-MER/HARDY-MER while the full text gives https://github.com/AI-S2-Lab/HARDY-MER. Please unify.
  2. [§4.5] The text says 'retrieved the top 1502 most similar samples' — this is likely a typo for 'top 150' or 'top 1500'; please correct.
  3. [Eq. (8)] The mutual information is computed via entropies H(f_p), H(f_q), H(f_{p,q}), but no estimator is specified for high-dimensional entropy. Since Eq. (8) is a central component of indirect hardness, a concrete estimation method (e.g., binning, kernel density, or a neural estimator) should be stated.
  4. [§5] The claim of being 'the first to integrate retrieval and curriculum learning' and 'the first work to apply RAG technology to multimodal emotion recognition' is stronger than what the cited literature supports; please soften or add a more precise comparison to prior retrieval-augmented training methods.
  5. [Table 3] The hyperparameter ablation explores only one parameter at a time and reports no significance. Since alpha1, alpha2, beta, and k are central to the dynamic curriculum, a sensitivity analysis with more values and multiple seeds would be more convincing.
  6. [§4.4] The 'w/o retrieval features' row removes all retrieval, effectively testing the whole retrieval curriculum; it cannot separate the effect of self-retrieval leakage from the benefit of external similar samples. This is related to Major Comment 1 and should be discussed.

Circularity Check

1 steps flagged · score 6.0 of 10

Training-time retrieval leaks the query's own full-modality record into the missing-modality input, so the central robustness gain is partly a train-only shortcut rather than a learned prediction.

  1. other [Section 3.3.1-3.3.3 (Feature Database Preparation; Hardness-based Dynamic Multimodal Features Retrieval; Retrieval-based Curriculum Training), Eq. (17)-(18)]
    "For each available modality, we query its corresponding FAISS index using the embedding z_m to retrieve the top-k most semantically similar samples, and record their indices. ... Based on these indices, we retrieve the corresponding multimodal features (acoustic, textual, and visual) from the three modality feature databases. ... Each training instance consists of the original input (x_a^miss, x_t, x_v) followed by its retrieved support samples ... During inference, we use the trained model to perform emotion prediction on inputs with missing modalities, without requiring dynamic curriculum re"

    The FAISS indexes are built from the training set's full three-modality features (Sec 3.3.1). A query's available-modality embeddings are themselves in the same index, so the top-1 retrieval is the query itself (zero L2 / maximal inner-product). Since k=5 and k'=ceil(h·k)>=1, every training instance always includes its own full feature vector as a support sample. That support is drawn from all three modality databases, so the modality marked 'missing' in the input is nevertheless present in the concatenated training context. The reconstruction/classification objective can then be minimized by copying the missing modality from the retrieved self-sample rather than by learning to predict it from available modalities. At inference retrieval is disabled, so the reported missing-modality gains

full rationale

No self-citation chain or imported uniqueness theorem is load-bearing; the comparisons are against external baselines and the final numbers are reported on held-out test sets. However, the retrieval database is constructed from the training data and the query is never excluded from the index, so every training sample retrieves itself as its top support. Because support features from all three modality databases are concatenated during training, the supposedly missing modality is present in the model's input whenever it is supposed to be absent. At inference, retrieval is removed. This is a concrete train/test input mismatch, not merely a hyperparameter-tuning concern. It means the central missing-modality robustness result is partially explained by an oracle shortcut. The 'w/o retrieval features' ablation removes retrieval entirely, so it cannot isolate or rule out the leakage. Hyperparameters alpha1, alpha2, beta, and k are tuned on the benchmark, but that is normal engineering and is not itself circular; the main circularity is the self-retrieval leakage that turns missing-modality training into full-modality training by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework introduces no physical entities. The central extras are four tuned hyperparameters and several domain assumptions about the validity of reconstruction error and mutual information as hardness signals. The ad hoc logistic combination and the retrieval-count formula are the most load-bearing free choices. The mutual information estimator, delegated to a cited method, adds another large unexamined component.

free parameters (4)
  • alpha1 = 0.6
    Weight on direct hardness in Eq. 10; sensitive to ablation in Table 3.
  • alpha2 = 0.4
    Weight on indirect hardness in Eq. 10; ablation shows performance depends on it.
  • beta = 4
    Scaling coefficient in the logistic function of Eq. 10; ablation with beta=2 and beta=8 changes results.
  • k = 5
    Maximum number of retrieved support samples in Eq. 18; set as a hyperparameter and not ablated.
assumptions (4)
  • standard math Mutual information identity I(X;Y)=H(X)+H(Y)-H(X,Y) holds for the entropy estimates used.
    Used in Eq. 4 and applied to fused features without a derivation of the entropy estimator.
  • domain assumption Zero vector is a valid representation for a missing modality and can be fed into encoders and mutual information computation.
    Introduced in Section 3.2.1, following prior work [18,23,55], with no analysis of its effect on hardness estimation.
  • domain assumption Reconstruction error (h_dir) and cross-modal mutual information (h_ind) jointly capture sample learning difficulty.
    The paper motivates this in Section 2.1 but does not validate the metric independently; the sign of h_ind is inconsistent with the stated motivation.
  • ad hoc to paper The unified hardness h = sigmoid(beta*(alpha1*hdir + alpha2*hind)) and the retrieval count k' = ceil(h*k) are appropriate curriculum controls.
    These formulas in Eqs. 10 and 18 are manually designed and tuned, with no derivation from first principles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalities." pith.science (2026). https://pith.science/paper/OMTTSFUS

@misc{pith2026250806800,
  author       = {Pith},
  title        = {Pith review of: Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OMTTSFUS}},
  note         = {Machine review of arXiv:2508.06800}
}
read the original abstract

Missing modalities have recently emerged as a critical research direction in multimodal emotion recognition (MER). Conventional approaches typically address this issue through missing modality reconstruction. However, these methods fail to account for variations in reconstruction difficulty across different samples, consequently limiting the model's ability to handle hard samples effectively. To overcome this limitation, we propose a novel Hardness-Aware Dynamic Curriculum Learning framework, termed HARDY-MER. Our framework operates in two key stages: first, it estimates the hardness level of each sample, and second, it strategically emphasizes hard samples during training to enhance model performance on these challenging instances. Specifically, we first introduce a Multi-view Hardness Evaluation mechanism that quantifies reconstruction difficulty by considering both Direct Hardness (modality reconstruction errors) and Indirect Hardness (cross-modal mutual information). Meanwhile, we introduce a Retrieval-based Dynamic Curriculum Learning strategy that dynamically adjusts the training curriculum by retrieving samples with similar semantic information and balancing the learning focus between easy and hard instances. Extensive experiments on benchmark datasets demonstrate that HARDY-MER consistently outperforms existing methods in missing-modality scenarios. Our code will be made publicly available at https://github.com/HARDY-MER/HARDY-MER.

Figures

Figures reproduced from arXiv: 2508.06800 by the authors.

Figure 1
Figure 1. Comparison between conventional paradigms for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of HARDY-MER consists of Multi-view Hardness Evaluation, Feature Database Preparation, Hardness [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. t-SNE visualizations for randomly selected samples in the IEMOCAP four-class across acoustic, textual, and visual [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Impact of different index construction methods on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 55 canonical work pages

  1. [1]

    Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. 2006. Greedy layer-wise training of deep networks. Advances in neural information processing systems 19 (2006)

    work page 2006

  2. [2]

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Ruther- ford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bog- dan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning . PMLR, 2206–2240

    2022

  3. [3]

    Brown, Henry L

    Peter C. Brown, Henry L. Roediger, and Mark A. McDaniel. 2014.Make It Stick: The Science of Successful Learning . Belknap Press: An Imprint of Harvard University Press, Cambridge, Massachusetts

    work page 2014

  4. [4]

    Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation 42 (2008), 335–359

    2008

  5. [5]

    Lei Cai, Zhengyang Wang, Hongyang Gao, Dinggang Shen, and Shuiwang Ji

    work page

  6. [6]

    Changde Du, Changying Du, Hao Wang, Jinpeng Li, Wei-Long Zheng, Bao- Liang Lu, and Huiguang He. 2018. Semi-supervised deep generative modelling of incomplete multi-modality emotional data. In Proceedings of the 26th ACM international conference on Multimedia . 108–116

    work page 2018

  7. [7]

    Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Ag- nieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al . 2016. Hybrid computing using a neural network with dynamic external memory. Nature 538, 7626 (2016), 471–476

    work page 2016

  8. [8]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning. PMLR, 3929–3938

    2020

Show all 63 references
  1. [9]

    Devamanyu Hazarika, Roger Zimmermann, and Soujanya Poria. 2020. Misa: Modality-invariant and-specific representations for multimodal sentiment analy- sis. In Proceedings of the 28th ACM international conference on multimedia . 1122– 1131

    2020

  2. [10]

    Jian Huang, Yanli Ji, Zhen Qin, Yang Yang, and Heng Tao Shen. 2023. Domi- nant SIngle-Modal SUpplementary Fusion (SIMSUF) For Multimodal Sentiment Analysis. IEEE Transactions on Multimedia (2023)

    2023

  3. [11]

    Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282 (2020)

    2020 arXiv

  4. [12]

    Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann. 2014. Easy samples first: Self-paced reranking for zero-example multimedia search. In Proceedings of the 22nd ACM international conference on Multimedia . 547–556

    2014

  5. [13]

    Lu Jiang, Deyu Meng, Shoou-I Yu, Zhenzhong Lan, Shiguang Shan, and Alexan- der Hauptmann. 2014. Self-paced learning with diversity. Advances in neural information processing systems 27 (2014)

    2014

  6. [14]

    Lul Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander Hauptmann

  7. [15]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing s...

    2020

  8. [16]

    Meng Li, Lin Wu, Arnold Wiliem, Kun Zhao, Teng Zhang, and Brian Lovell. 2019. Deep instance-level hard negative mining model for histopathology images. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, Oc...

    2019

  9. [17]

    Tian-Bao Li, An-An Liu, Dan Song, Wen-Hui Li, Xuan-Ya Li, and Yu-Ting Su. 2023. Focus on hard samples: Hierarchical unbiased constraints for cross-domain 3D model retrieval. IEEE Transactions on Circuits and Systems for Video Technology 33, 11 (2023), 7036–7049

    2023

  10. [18]

    Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. 2023. GCNet: Graph completion network for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intelligence 45, 7 (2023), 8419–8432

    2023

  11. [19]

    Wei-Cheng Lin, Lucas Goncalves, and Carlos Busso. 2023. Enhancing Resilience to Missing Data in Audio-Text Emotion Recognition with Multi-Scale Chunk Regularization. In Proceedings of the 25th International Conference on Multimodal Interaction. 207–215

    2023

  12. [20]

    Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. 2021. Completer: Incomplete multi-view clustering via contrastive prediction. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition . 11174–11183

    2021

  13. [21]

    Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, et al. 2024. Rho-1: Not all tokens are what you need. arXiv preprint arXiv:2404.07965 (2024)

    2024 arXiv

  14. [22]

    Rui Liu, Zhenqi Jia, Feilong Bao, and Haizhou Li. 2025. Retrieval-Augmented Dialogue Knowledge Aggregation for expressive conversational speech synthesis. Information Fusion (2025), 102948

    2025

  15. [23]

    Rui Liu, Haolin Zuo, Zheng Lian, Bjorn W Schuller, and Haizhou Li. 2024. Con- trastive Learning based Modality-Invariant Feature Acquisition for Robust Mul- timodal Emotion Recognition with Missing Modalities. IEEE Transactions on Affective Computing (2024)

    2024

  16. [24]

    Wei Luo, Mengying Xu, and Hanjiang Lai. 2023. Multimodal reconstruct and align net for missing modality problem in sentiment analysis. In International Conference on Multimedia Modeling . Springer, 411–422

    2023

  17. [25]

    Sijie Mai, Haifeng Hu, and Songlong Xing. 2020. Modality to modality translation: An adversarial representation learning and graph fusion network for multimodal fusion. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 164–172

    2020

  18. [26]

    Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 6818–6825

    2019

  19. [27]

    Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabás Poczós, and Tom Mitchell. 2019. Competence-based Curriculum Learning for Neural Machine Translation. In Proceedings of the 2019 Conference of the North Ameri- can Chapter of the Association for Computational...

    2019

  20. [28]

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition . 815–823

    2015

  21. [29]

    Ashish Seth, Ramaneswaran Selvakumar, S Sakshi, Sonal Kumar, Sreyan Ghosh, and Dinesh Manocha. 2024. EH-MAM: Easy-to-Hard Masked Acoustic Modeling for Self-Supervised Speech Representation Learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language...

    2024

  22. [30]

    Guangyao Shen, Xin Wang, Xuguang Duan, Hongzhi Li, and Wenwu Zhu. 2020. Memor: A dataset for multimodal emotion reasoning in videos. In Proceedings of the 28th ACM International Conference on Multimedia . 493–502

    2020

  23. [31]

    Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. 2016. Training region- based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition . 761–769

    2016

  24. [32]

    Qiya Song, Jiajun Hu, Lin Xiao, Bin Sun, Xieping Gao, and Shutao Li. 2025. Diffcl: A diffusion-based contrastive learning framework with semantic alignment for multimodal recommendations. IEEE Transactions on Neural Networks and Learning Systems (2025)

    2025

  25. [33]

    Qiya Song, Bin Sun, and Shutao Li. 2022. Multimodal sparse transformer network for audio-visual speech recognition. IEEE Transactions on Neural Networks and Learning Systems 34, 12 (2022), 10028–10038

    2022

  26. [34]

    Haoqin Sun, Shiwan Zhao, Shaokai Li, Xiangyu Kong, Xuechen Wang, Jiaming Zhou, Aobo Kong, Yong Chen, Wenjia Zeng, and Yong Qin. 2025. Enhancing Emotion Recognition in Incomplete Data: A Novel Cross-Modal Alignment, Re- construction, and Refinement Framework. InICASSP 2025-2025...

    2025

  27. [35]

    Jiajia Tang, Kang Li, Xuanyu Jin, Andrzej Cichocki, Qibin Zhao, and Wanzeng Kong. 2021. CTFN: Hierarchical learning for multimodal sentiment analysis using coupled-translation fusion network. In Proceedings of the 59th Annual Meeting of the Association for Computational Lingui...

    2021

  28. [36]

    Wenhao Tang, Sheng Huang, Xiaoxian Zhang, Fengtao Zhou, Yi Zhang, and Bo Liu. 2023. Multiple instance learning framework with masked hard instance mining for whole slide image classification. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision . 4078–4...

    2023

  29. [37]

    Mani Kumar Tellamekala, Shahin Amiriparian, Björn W Schuller, Elisabeth André, Timo Giesbrecht, and Michel Valstar. 2023. COLD fusion: Calibrated and ordinal latent distribution fusion for uncertainty-aware multimodal emotion recognition. IEEE Transactions on Pattern Analysis ...

    2023

  30. [38]

    Juan Vazquez-Rodriguez, Grégoire Lefebvre, Julien Cumin, and James L Crowley

  31. [39]

    Haochen Wang, Kaiyou Song, Junsong Fan, Yuxi Wang, Jin Xie, and Zhaoxiang Zhang. 2023. Hard patches mining for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10375– 10385

    2023

  32. [40]

    Kai Wang, Yizhou Peng, Hao Huang, Ying Hu, and Sheng Li. 2022. Mining hard samples locally and globally for improved speech separation. InICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 6037–6041

    2022

  33. [41]

    Keze Wang, Xiaopeng Yan, Dongyu Zhang, Lei Zhang, and Liang Lin. 2018. Towards human-machine cooperation: Self-supervised sample mining for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1605–1613

    2018

  34. [42]

    Ning Wang, Hui Cao, Jun Zhao, Ruilin Chen, Dapeng Yan, and Jie Zhang. 2022. M2R2: Missing-Modality Robust emotion Recognition framework with iterative data augmentation. IEEE Transactions on Artificial Intelligence 4, 5 (2022), 1305– 1316

    2022

  35. [43]

    Xin Wang, Yudong Chen, and Wenwu Zhu. 2021. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence 44, 9 (2021), 4555–4576

    2021

  36. [44]

    Yuanzhi Wang, Yong Li, and Zhen Cui. 2024. Incomplete multimodality-diffused emotion recognition. Advances in Neural Information Processing Systems 36 (2024)

    2024

  37. [45]

    Yulin Wang, Yang Yue, Rui Lu, Tianjiao Liu, Zhao Zhong, Shiji Song, and Gao Huang. 2023. Efficienttrain: Exploring generalized curriculum learning for train- ing visual backbones. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5852–5864

    2023

  38. [46]

    Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Jiashi Feng, Yao Zhao, and Shuicheng Yan. 2016. Stc: A simple to complex framework for weakly-supervised semantic segmentation. IEEE transactions on pattern analysis and machine intelligence 39, 11 (2016)...

    2016

  39. [47]

    Lirong Wu, Yunfan Liu, Yufei Huang, Haitao Lin, Cheng Tan, and Stan Z. Li

  40. [48]

    Shangyu Wu, Ying Xiong, Yufei Cui, Haolun Wu, Can Chen, Ye Yuan, Lianming Huang, Xue Liu, Tei-Wei Kuo, Nan Guan, et al. 2024. Retrieval-augmented gener- ation for natural language processing: A survey. arXiv preprint arXiv:2407.13193 (2024)

    2024 arXiv

  41. [49]

    Yuxin Wu and Yuandong Tian. 2022. Training agent for first-person shooter game with actor-critic curriculum learning. In International Conference on Learning Representations

    2022

  42. [50]

    Wenxin Xu, Hexin Jiang, and Xuefeng Liang. 2024. Leveraging Knowledge of Modality Experts for Incomplete Multimodal Learning. In Proceedings of the 32nd ACM International Conference on Multimedia . 438–446

    2024

  43. [51]

    Ziqi Yuan, Wei Li, Hua Xu, and Wenmeng Yu. 2021. Transformer-based feature reconstruction network for robust multimodal sentiment analysis. In Proceedings of the 29th ACM International Conference on Multimedia . 4400–4407

    2021

  44. [52]

    Ziqi Yuan, Yihe Liu, Hua Xu, and Kai Gao. 2023. Noise imitation based adver- sarial training for robust multimodal sentiment analysis. IEEE Transactions on Multimedia 26 (2023), 529–539

    2023

  45. [53]

    Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, and Qinghua Hu. 2020. Deep partial multi-view learning. IEEE transactions on pattern analysis and machine intelligence 44, 5 (2020), 2402–2415

    2020

  46. [54]

    Fei Zhao, Chunhui Li, Zhen Wu, Yawen Ouyang, Jianbing Zhang, and Xinyu Dai

  47. [55]

    Jinming Zhao, Ruichen Li, and Qin Jin. 2021. Missing modality imagination net- work for emotion recognition with uncertain missing modalities. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference...

    2021

  48. [56]

    Jianing Zhou, Ziheng Zeng, and Suma Bhat. 2023. CLCL: Non-compositional expression detection with contrastive learning and curriculum learning. In Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 730–743

    2023

  49. [57]

    Yuchen Zhou, Guang Tan, Mengtang Li, and Chao Gou. 2023. Learning from easy to hard pairs: Multi-step reasoning network for human-object interaction detection. In Proceedings of the 31st ACM International Conference on Multimedia . 4368–4377

    2023

  50. [58]

    Haolin Zuo, Rui Liu, Jinming Zhao, Guanglai Gao, and Haizhou Li. 2023. Ex- ploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

    2023

  51. [59]

    In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    M2DF: Multi-grained Multi-curriculum Denoising Framework for Multi- modal Aspect-based Sentiment Analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . 9057–9070

    2023

  52. [2015]

    In Proceedings of the AAAI Conference on Artificial Intelligence, Vol

    Self-paced curriculum learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29

  53. [2018]

    In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining

    Deep adversarial learning for multi-modality missing data completion. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining . 1158–1166

  54. [2023]

    In2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII)

    Accommodating Missing Modalities in Time-Continuous Multimodal Emotion Recognition. In2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 1–8

  55. [2024]

    https://openreview.net/forum?id=X6ajk22thA

    HGMD: Rethinking Hard Sample Distillation for GNN-to-MLP Knowledge Distillation. https://openreview.net/forum?id=X6ajk22thA

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.