Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection

Han Wang; Roy Ka-Wei Lee; Rui Yang Tan

REVIEW 3 major objections 5 minor 45 references

Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Meme datasets, after label re-alignment, can substitute for and augment video data in hateful-video detection training, outperforming existing benchmarks.

desk verdict Plausible idea and useful released resources, but the evaluation is compromised by test-set selection of epoch, few-shot N, and dataset size, so the headline gains over SOTA are not trustworthy yet. read the letter →

arxiv 2501.15438 v1 pith:AJHBQ5CN submitted 2025-01-26 cs.CV cs.MM

Han Wang , Rui Yang Tan , Roy Ka-Wei Lee This is my paper · ORCID

classification cs.CVcs.MM

keywords cross-modaltransferhatefulvideodetectionmemedatasetlabelalignmentre-annotationvision-languagemodelsdatascarcitymultimodalhatespeech

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that hateful meme datasets, once their labels are re-aligned to the definitions used by video hate-speech benchmarks, can stand in for scarce video data when training hateful-video detection models. The authors argue this matters because video annotation is expensive and video datasets are small, while meme datasets are larger and cheaper to label. They introduce a human-assisted re-annotation pipeline that combines the original meme label, a vision-language model's few-shot prediction, and a human annotation in a majority vote, then fine-tune vision-language models on the re-annotated memes. On the MHC and HateMM benchmarks, fine-tuning on re-annotated memes alone performs comparably to fine-tuning on the video data itself, and adding the re-annotated memes to the video training set improves Macro-F1 over both the video-only baseline and the best results reported in the original dataset papers.

What carries the argument

The load-bearing mechanism is a model-prediction-driven, human-assisted re-annotation pipeline that harmonizes label definitions across datasets. For each meme, the pipeline takes the original dataset label, a prediction from LLaMA-3.2-11B prompted with a few video demonstration examples, and a human annotation, then assigns the label by majority vote. The re-annotated memes are then used to fine-tune a vision-language model via Low-Rank Adaptation (LoRA) adapters on the query and value layers of the attention mechanism, with video frames sampled as single images for the image-based model or as 16 augmented frames for the video-based model. This machinery is what converts a mismatched label space into a transferable training signal.

What would settle it

A direct test would be to re-annotate the same meme datasets with human annotators only, following the video datasets' annotation guidelines, and compare RM-FT performance against that human-only re-annotation; if the model-inclusive majority vote does not match or beat the human-only version, the reported gains could be partly an artifact of the model fine-tuning on its own outputs. Reporting inter-annotator agreement and the fraction of memes that required human adjudication would let a reader check this directly.

Watch

Extended reading notes

Core claim

The paper's central claim is that cross-modal transfer from memes to videos works once label definitions are aligned. With the re-annotated Facebook Hateful Memes (FHM) dataset and LLaMA-3.2-11B, fine-tuning on memes alone (RM-FT) reaches Macro-F1 0.80 on MHC, exceeding fine-tuning on the MHC videos themselves (Vid-FT, 0.78), and on HateMM it matches Vid-FT at 0.81. Combining re-annotated memes with video data (Vid+RM-FT) yields Macro-F1 0.81 on MHC and 0.82 on HateMM, surpassing the best results reported by the MHC paper (0.79) and the HateMM paper (0.79), as well as the video-only fine-tuning baselines. The authors interpret this as evidence that meme data can substitute for video data in resource-scarce settings and that augmentation with memes provides complementary signal beyond what video-only training captures.

Load-bearing premise

The re-annotation pipeline assumes that the majority vote among the original meme label, a vision-language model's few-shot prediction, and a single human annotation produces labels that faithfully match the video datasets' definitions, without reporting how often the model prediction dominates or how reliable the human adjudication is.

Editorial extensions

If this is right

In settings where hateful videos are scarce or expensive to annotate, models can be trained on re-annotated memes and then applied to video classification with little or no performance loss.
Augmenting small video datasets with re-annotated memes yields consistent Macro-F1 improvements on both MHC and HateMM, offering a cheap way to push benchmark performance.
The transfer works across two different vision-language model architectures, an image-based model and a video-based model, though the larger image-based model gains more from the meme data.
The released re-annotated meme datasets and code give other researchers a ready-made resource for training or validation in video hate speech tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the label-alignment step is the true driver, then similar cross-modal transfer should work from other image-text datasets (e.g., cyberbullying or misogyny memes) to videos in those specific domains, where the label definitions already overlap.
The majority-vote design may partly train the model on its own outputs: when the model's prediction agrees with the original label, no human review is triggered, so the model's biases could be reinforced; measuring agreement rates per source would clarify how independent the final labels are.
The observed plateau in gains beyond roughly 3,000 memes could be a model-capacity effect; a larger base model might convert the same meme data into larger improvements, and that is directly testable.
A natural extension would be to apply the same re-annotation strategy to align the two video datasets' label definitions to each other, enabling cross-video transfer without any meme data.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is genuinely worth a look: use meme data, after a label-alignment reannotation step, as a substitute or augmentation for scarce hateful video datasets. That specific transfer path (meme-to-video) is not in the cited literature, and the released reannotated meme datasets and code are a real community resource. The writing is clear, the problem is motivated, and the authors are upfront about possible data leakage in their zero-shot results. Credit where due: this is a sensible, cheap-data strategy, and the qualitative analysis of label discrepancies between FHM/MAMI and the video datasets is useful.

The soft spots are substantial and mostly in the evaluation protocol. The paper explicitly selects the best-performing epoch on the video test set (Section 4.1.3), chooses few-shot N using the MHC/HateMM test sets (Table 1), and picks meme/video dataset sizes by testing on MHC (Table 2). With one run per configuration and no error bars or seeds, the reported Macro-F1 gains of 0.01–0.03 over Vid-FT, and 0.81/0.82 vs SOTA 0.79/0.79, could be selection noise. The stress-test note is correct: this is the load-bearing flaw, not a side issue. The reannotation circularity is real but secondary; the model family used for majority-vote prediction is the same one later fine-tuned, and there is no human-only reannotation baseline, so part of RM-FT's advantage over OM-FT may be self-training. That concern is moderate, not fatal, because the majority vote includes the original label and a human, but the absence of inter-annotator agreement and counts of human adjudication makes it impossible to judge how much model self-bias leaked in.

Who is this for? Researchers working on multimodal hate detection and dataset-efficient transfer. The idea is worth building on, and the resources are reusable. But the quantitative claims need a rerun with a held-out validation split for epoch and hyperparameter selection, multiple seeds, and a human-only reannotation comparison before I'd trust the SOTA-beating numbers. This deserves a serious referee—the topic matters and the method is plausible—but it should come back requiring major revision on the evaluation protocol.

Referee Report

3 major / 5 minor

Summary. The paper proposes cross-modal transfer from hateful meme datasets to hateful video detection, motivated by the scarcity and high annotation cost of video datasets. A re-annotation pipeline aligns meme labels with video dataset definitions by majority voting among the original label, a few-shot LLaMA-3.2-11B prediction, and a human annotation. The authors fine-tune LLaMA-3.2-11B and LLaVA-NeXT-Video-7B with LoRA under five strategies (No FT, Vid-FT, OM-FT, RM-FT, Vid+RM-FT) and report that re-annotated memes can substitute for video data and that augmenting video data with re-annotated memes outperforms video-only fine-tuning and prior state-of-the-art results on MHC and HateMM. They release the re-annotated meme datasets and code.

Significance. If the results are valid, the approach is practically significant: meme annotation is substantially cheaper than video annotation, and a successful substitution/augmentation strategy would lower the barrier for hateful video detection. The paper includes useful strengths: public release of the re-annotated datasets and code, evaluation with two different VLM families, and an explicit acknowledgment of potential data leakage for HateMM. However, the central claims rest on an evaluation protocol that selects reporting epochs, few-shot counts, and dataset sizes on the same test sets used for comparison, and the re-annotation loop involves the same model family that is later fine-tuned. These issues must be resolved before the SOTA-beating numbers can be taken as evidence for cross-modal transfer.

major comments (3)

[§4.1.3, §4.2, §3.2.2, Tables 1–4] The evaluation protocol selects hyperparameters and reporting choices on the test sets used for the headline comparisons. Section 4.1.3 states that 'The best-performing epoch on the video test data set was selected to report the results'; Section 4.2 chooses the meme and video dataset sizes n by testing on the MHC test set; and Section 3.2.2 chooses the few-shot count N using the MHC and HateMM test sets. Each configuration is run once and no error bars, seeds, or confidence intervals are reported. The reported improvements (e.g., Vid+RM-FT Macro-F1 0.81 on MHC vs. 0.79 from the MHC paper, and +0.01–0.03 over Vid-FT in Table 4) may therefore be selection noise rather than evidence that re-annotated memes transfer. The central substitution/augmentation claim is not supported until the protocol is re-run with a held-out validation split used for epoch, N, and n selection, and with variability reported.
[§3.2.1–§3.2.3] The re-annotation labels are produced by majority voting among the original meme label, a LLaMA-3.2-11B prediction, and a human annotation, and the same model family (LLaMA-3.2-11B) is later fine-tuned and evaluated on the video test sets. The paper reports no inter-annotator agreement, no count of how many memes required human adjudication, and no comparison against a purely human-reannotated dataset. Consequently, the observed RM-FT gains over OM-FT in Table 3 could partly reflect the model being trained on its own outputs rather than on genuinely aligned labels. The authors should report the distribution of majority-vote outcomes, agreement statistics, and a human-only re-annotation condition to support the claim that the pipeline improves label alignment.
[§3.2.2] The paper itself notes that 'the strong zero-shot performance could be due to potential data leakage, as LLaMA-3.2-11B was released after the HateMM dataset,' and that this possibility cannot be ruled out. Since the same model is used for few-shot demonstration selection and fine-tuning on HateMM, the HateMM results in Tables 1, 3, and 4 may be inflated. The authors should either use a model whose training data provably excludes HateMM, evaluate on a split released after the model's cutoff, or provide a quantitative bound on the potential leakage effect. The current acknowledgment is not sufficient to support the state-of-the-art claim on HateMM.

minor comments (5)

[§3.3] The sentence 'We employ Low-Rank Adaptation (LoRA) [22] adapters to fine-tune the pre-trained VLM during.' ends with the dangling word 'during'; it should read 'during fine-tuning.'
[References] Reference [43] cites 'arXiv preprint arXiv:2407.xxxxx'; the full arXiv identifier and title should be provided.
[Table 3] The table header uses 'O' for MHC and 'H' for HateMM; the text explains that MHC merges hateful and offensive into a single offensive class, but the table would be clearer if the label definitions were restated in the caption.
[§4.2] The annotation-time estimates in Section 4.2 do not specify whether '0.5 minutes per meme' and '2 minutes per video' are per annotator, nor how many annotators were involved; please clarify to make the cost comparison reproducible.
[§3.2.2] For LLaVA-NeXT-Video-7B, the paper replaces visual input in demonstrations with 'vision-based descriptions of video content provided by human annotators'; it is unclear whether these are the same annotators as in the majority-voting step and whether this description effort is included in the reported annotation cost.

Circularity Check

2 steps flagged · score 6.0 of 10

Re-annotation labels are co-produced by the same LLaMA-3.2-11B later fine-tuned, and test-set selection of epoch and dataset sizes inflates the reported SOTA gains; partial circularity.

fitted input called prediction [Section 3.2.1-3.2.2 (Re-Annotation Process and Model Prediction)]
"The final label for each meme is determined by majority voting among the three sources: dataset label, model prediction, and human annotation. ... For the re-annotation of the meme dataset, we utilized the optimal N value of video demonstrations and selected LLaMA-3.2-11B as the prediction model to generate labels."

LLaMA-3.2-11B contributes the 'model prediction' vote that co-determines the re-annotated meme labels, and the same LLaMA-3.2-11B is then LoRA-fine-tuned on those labels and evaluated on video test sets. The training target is partly the model's own output, so RM-FT gains over OM-FT can reflect the model being aligned to its own few-shot predictions (which were conditioned on video demonstrations) rather than to an independent video-label definition. Without a human-only re-annotation baseline or counts of how often the model vote changed the label, the re-annotation loop is not shown to be an external ground truth.
fitted input called prediction [Section 4.1.3 and 4.2 (Fine-Tuning Strategies and Hyperparameter Analysis)]
"The best-performing epoch on the video test data set was selected to report the results. ... Fine-tuning experiments were conducted using LLaMA-3.2-11B on the re-annotated FHM and MHC datasets, with testing performed on the MHC dataset."

The reported Macro-F1 values are not out-of-sample predictions: the epoch (4.1.3), the meme/video dataset sizes n (4.2), and the few-shot count N (Table 1) are all chosen by optimizing on the same MHC/HateMM test sets that are later used for the SOTA comparison. Each configuration is run once with no error bars or seeds, so the small gains (e.g., Vid+RM-FT 0.81 vs MHC paper 0.79) are partly selection noise rather than evidence that memes transfer. The test-set-optimized numbers are then presented as the method's performance, which is a fitted input masquerading as a prediction.

full rationale

The paper's central cross-modal transfer idea is not inherently circular: meme images and video frames are distinct, and the video test labels are external human annotations. However, two load-bearing steps compromise the derivation. First, the re-annotation pipeline defines meme labels using a majority vote that includes predictions from LLaMA-3.2-11B, the very model later fine-tuned on those labels and tested on videos. This creates a self-training loop whose independence is unverified, because the paper reports no human-only reannotation baseline, no inter-annotator agreement, and no count of how often the model vote changed the final label. Second, the evaluation protocol selects the best epoch on the video test set, chooses dataset sizes by testing on the MHC test set, and selects the few-shot demonstration count using the same test datasets, then presents the resulting Macro-F1 values as the method's performance. The reported gains over SOTA and over Vid-FT are therefore at least partly artifacts of test-set optimization rather than out-of-sample evidence that re-annotated memes substitute for or augment video data. These issues make the central claim partially circular, though not fully definitional, so a score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on test-set-tuned hyperparameters (few-shot N and dataset sizes n), label-mapping decisions for each dataset pair, and the quality of the model-assisted re-annotation. The paper introduces no new entities or forces. The key load-bearing choices are the label remappings and the majority-voting reannotation, whose human component is under-specified.

free parameters (4)

Number of few-shot demonstrations N = N=2 for MHC and LLaMA-3.2-11B; N=0 for HateMM (Table 1)
Selected by best Macro-F1 on the test sets in Table 1, making it a test-set-tuned hyperparameter.
Fine-tuning dataset sizes n = 800 videos, 3,000 memes
Section 4.2 chooses these sizes based on best MHC test performance, so they are fitted to test data.
Random single-frame sampling for LLaMA-3.2-11B = not specified
Section 3.3 randomly samples one frame per video; without a fixed seed, results are stochastic and not fully reproducible.
LoRA rank and alpha = not specified
LoRA adapters are used on query and value layers, but the rank and scaling parameters are not reported, hindering exact reproduction.

assumptions (4)

domain assumption MHC hateful and offensive labels are merged into a single offensive class for binary classification
Section 3.1 performs this remapping; it changes the task definition and affects comparability with the original MHC benchmark.
domain assumption LLaMA-3.2-11B zero-shot predictions are not inflated by training on HateMM
The paper acknowledges this risk in Section 3.2.2 but still uses the model's predictions for re-annotation and baselines.
domain assumption Human annotators in the re-annotation pipeline provide labels consistent with video definitions and are free of systematic bias
The pipeline (Section 3.2) relies on human adjudication without reporting annotator count, expertise, or inter-annotator agreement.
domain assumption A single randomly sampled frame adequately represents a video's visual content for LLaMA-3.2-11B
Section 3.3 samples one frame per video, discarding temporal structure; the evaluation assumes this suffices for hate detection.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection." pith.science (2026). https://pith.science/paper/AJHBQ5CN

@misc{pith2026250115438,
  author       = {Pith},
  title        = {Pith review of: Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJHBQ5CN}},
  note         = {Machine review of arXiv:2501.15438}
}

read the original abstract

Detecting hate speech in online content is essential to ensuring safer digital spaces. While significant progress has been made in text and meme modalities, video-based hate speech detection remains under-explored, hindered by a lack of annotated datasets and the high cost of video annotation. This gap is particularly problematic given the growing reliance on large models, which demand substantial amounts of training data. To address this challenge, we leverage meme datasets as both a substitution and an augmentation strategy for training hateful video detection models. Our approach introduces a human-assisted reannotation pipeline to align meme dataset labels with video datasets, ensuring consistency with minimal labeling effort. Using two state-of-the-art vision-language models, we demonstrate that meme data can substitute for video data in resource-scarce scenarios and augment video datasets to achieve further performance gains. Our results consistently outperform state-of-the-art benchmarks, showcasing the potential of cross-modal transfer learning for advancing hateful video detection. Dataset and code are available at https://github.com/Social-AI-Studio/CrossModalTransferLearning.

Figures

Figures reproduced from arXiv: 2501.15438 by the authors.

**Figure 2.** Re-annotation pipeline of meme datasets. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Meme count distribution for FHM dataset, catego [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Meme count distribution for MAMI dataset, cate [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 34 canonical work pages

[1]

Alcântara, V

C. Alcântara, V. Moreira, and D. Feijo. 2020. Offensive video detection: dataset and baseline results. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 4309–4319

work page 2020
[2]

Md Rabiul Awal, Rui Cao, Roy Ka-Wei Lee, and Sandra Mitrović. 2021. Angry- bert: Joint learning target and emotion for hate speech detection. In Pacific-Asia conference on knowledge discovery and data mining . Springer, 701–713

work page 2021
[3]

Md Rabiul Awal, Roy Ka-Wei Lee, Eshaan Tanwar, Tanmay Garg, and Tanmoy Chakraborty. 2023. Model-agnostic meta-learning for multilingual hate speech detection. IEEE Transactions on Computational Social Systems 11, 1 (2023), 1086– 1095

work page 2023
[4]

Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is space-time atten- tion all you need for video understanding?. In Proceedings of the International Conference on Machine Learning (ICML) , Vol. 2. 4

work page 2021
[5]

R. Cao, M. S. Hee, A. Kuek, W. H. Chong, R. K. W. Lee, and J. Jiang. 2023. Pro- cap: Leveraging a frozen vision-language model for hateful meme detection. In Proceedings of the 31st ACM International Conference on Multimedia . 5244–5252

work page 2023
[6]

Rui Cao and Roy Ka-Wei Lee. 2020. Hategan: Adversarial generative-based data augmentation for hate speech detection. In Proceedings of the 28th International Conference on Computational Linguistics . 6327–6338

work page 2020
[7]

R. Cao, R. K. W. Lee, W. H. Chong, and J. Jiang. 2023. Prompting for Multimodal Hateful Meme Classification. arXiv preprint arXiv:2302.04156 (2023)

arXiv 2023
[8]

Rui Cao, Roy Ka-Wei Lee, and Tuan-Anh Hoang. 2020. DeepHate: Hate speech detection via multi-faceted text representations. In Proceedings of the 12th ACM Conference on Web Science. 11–20

work page 2020

Show all 45 references

[9]

R. Cao, R. K. W. Lee, and J. Jiang. 2024. Modularized Networks for Few-shot Hateful Meme Detection. In Proceedings of the ACM on Web Conference 2024 . 4575–4584

2024
[10]

João Carreira and Andrew Zisserman. 2017. Temporal 3D ConvNets: New Archi- tecture and Transfer Learning for Video Classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 6568–6577

2017
[11]

Chhabra and D

A. Chhabra and D. K. Vishwakarma. 2023. A literature survey on multimodal and multilingual automatic hate speech identification. Multimedia Systems 29, 3 (2023), 1203–1230

2023
[12]

M. Das, R. Raj, P. Saha, B. Mathew, M. Gupta, and A. Mukherjee. 2023. Hatemm: A Multi-Modal Dataset for Hate Video Classification. In Proceedings of the Inter- national AAAI Conference on Web and Social Media , Vol. 17. 1014–1023

2023
[13]

Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 11. 512–515

2017
[14]

Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. 2018. Hate Speech Dataset from a White Supremacy Forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2) , Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, an...

2018 doi
[15]

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recogn...

2021
[16]

Dubey, A

A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, and R. Gana- pathy. 2024. The LLAMA 3 Herd of Models. arXiv preprint arXiv:2407.21783 (2024)

2024 arXiv
[17]

Elisabetta Fersini, Francesca Gasparini, Giuliano Rizzi, Andrea Saibene, Borja Chulvi, Paolo Rosso, and Jonas Sorensen. 2022. SemEval-2022 Task 5: Multime- dia Automatic Misogyny Identification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-...

2022
[18]

Elisabetta Fersini, Debora Nozza, and Paolo Rosso. 2018. Overview of the EVALITA 2018 task on automatic misogyny identification (AMI). In CEUR Work- shop Proceedings, Vol. 2263. 1–9

2018
[19]

Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context- aware models. arXiv preprint arXiv:1710.07395 (2017)

2017 arXiv
[20]

M. S. Hee, A. Kumaresan, and R. K. W. Lee. 2024. Bridging Modalities: Enhancing Cross-Modality Hate Speech Detection with Few-Shot In-Context Learning.arXiv preprint arXiv:2410.05600 (2024)

2024 arXiv
[21]

M. S. Hee, S. Sharma, R. Cao, P. Nandi, P. Nakov, T. Chakraborty, and R. Lee. 2024. Recent Advances in Online Hate Speech Moderation: Multimodality and the Role of Large Models. In Findings of the Association for Computational Linguistics: EMNLP 2024. 4407–4419

2024
[22]

Edward Hu, Xuezhi Peng, Yi Li, Xifeng Liu, Jie He, Ziyang Chen, Ziyi Li, Yiming Zhang, Caiming Xiong, and Kai-Wei Chang. 2021. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the 39th International Conference on Machine Learning (ICML). https://arxiv.org/...

2021 arXiv
[23]

Mohan, Ankur

Douwe Kiela, Hamed Firooz, and et al. Mohan, Ankur. 2020. The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes. Advances in Neural Information Processing Systems 33 (2020), 2611–2624

2020
[24]

Dong-Ho Lee, Hyundong Cho, Woojeong Jin, Jihyung Moon, Sungjoon Park, Paul Röttger, Jay Pujara, and Roy Ka-Wei Lee. 2024. Improving covert toxicity detection by retrieving and generating references. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024) . 266–274

2024
[25]

Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. 2022. Video swin transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 3202–3211

2022
[26]

Koushik Maity, Pankaj Jha, Subhasis Saha, and Pawan Bhattacharyya. 2022. A multitask framework for sentiment, emotion and sarcasm aware cyberbullying detection from multi-modal code-mixed memes. In Proceedings of the 45th In- ternational ACM SIGIR Conference on Research and De...

2022
[27]

L. H. X. Ng, A. X. W. Lim, and R. K. W. Lee. 2024. Love-Hate Dataset: A Multi- Modal Multi-Platform Dataset Depicting Emotions in the 2023 Israel-Hamas War. In Companion Proceedings of the ACM on Web Conference 2024 . 1807–1815

2024
[28]

Ri Chi Ng, Nirmalendu Prakash, Ming Shan Hee, Kenny Tsu Wei Choo, and Roy Ka-Wei Lee. 2024. SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singapore. arXiv preprint arXiv:2405.01842 (2024)

2024 arXiv
[29]

J. Pan, Z. Lin, X. Zhu, J. Shao, and H. Li. 2022. ST-Adapter: Parameter-efficient Image-to-video Transfer Learning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 35. 26462–26477

2022
[30]

Souvik Pramanick, Dimitar Dimitrov, Ritam Mukherjee, Shubham Sharma, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. Detecting Harmful Memes and Their Targets. arXiv preprint arXiv:2110.00413 (2021)

2021 arXiv
[31]

Aminul-Haq Razavi, Diana Inkpen, Sasha Uritsky, and Stan Matwin. 2010. Offen- sive Language Detection Using Multi-level Classification. Advances in Artificial Intelligence 6085 (2010), 16–27

2010
[32]

Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detec- tion using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media . 1–10

2017
[33]

H. Wang, M. S. Hee, M. R. Awal, K. T. W. Choo, and R. K. W. Lee. 2023. Evaluating GPT-3 generated explanations for hateful content moderation. arXiv preprint arXiv:2305.17680 (2023)

2023 arXiv
[34]

Han Wang, T. R. Yang, U. Naseem, and Roy Ka-Wei Lee. 2024. Multihateclip: A multilingual benchmark dataset for hateful video detection on YouTube and Bilibili. In Proceedings of the 32nd ACM International Conference on Multimedia . 7493–7502

2024
[35]

William Warner and Julia Hirschberg. 2012. Detecting hate speech on the World Wide Web. In Proceedings of the second workshop on language in social media . 19–26

2012
[36]

Zeerak Waseem. 2016. Are You a Racist or Am I Seeing Things? Annotator Influ- ence on Hate Speech Detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science , David Bamman, A. Seza Doğruöz, Jacob Eisenstein, Dirk Hovy, David Jurgens, Bre...

2016 doi
[37]

Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In Proceedings of the NAACL Student Research Workshop. 88–93

2016
[38]

Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predic- tive Features for Hate Speech Detection on Twitter. In Proceedings of the NAACL Student Research Workshop, Jacob Andreas, Eunsol Choi, and Angeliki Lazaridou (Eds.). Association for Computational Lin...

2016 doi
[39]

C. S. Wu and U. Bhandary. 2020. Detection of hate speech in videos using machine learning. In 2020 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, 585–590

2020
[40]

Yunze Xiao, Yujia Hu, Kenny Tsu Wei Choo, and Roy Ka-wei Lee. 2024. Toxi- CloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations. arXiv preprint arXiv:2406.12223 (2024)

2024 arXiv
[41]

H. Yao, W. Wu, and Z. Li. 2023. Side4video: Spatial-temporal side net- work for memory-efficient image-to-video transfer learning. arXiv preprint arXiv:2311.15769 (2023)

2023 arXiv
[42]

Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the Type and Target of Offensive Posts in Social Media. In Proceedings of the NAACL-HLT 2019, Volume 1 (Long and Short Papers). 1415–1420. Conference acronym ’XX, J...

2019
[43]

Zhang, B

Y. Zhang, B. Li, H. Liu, Y. Lee, L. Gui, D. Fu, and C. Li. 2024. Llava-next: A Strong Zero-shot Video Understanding Model. arXiv preprint arXiv:2407.xxxxx (2024)

2024
[44]

Ziyi Zhang, Derek Robinson, and John Tepper. 2018. Detecting hate speech on Twitter using a convolution-GRU based deep neural network. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings, Vol. 15. Springer Intern...

2018
[45]

J. Zhu, R. K. W. Lee, and W. H. Chong. 2022. Multimodal Zero-Shot Hateful Meme Detection. In Proceedings of the 14th ACM Web Science Conference 2022 . 382–389

2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Alcântara, V

C. Alcântara, V. Moreira, and D. Feijo. 2020. Offensive video detection: dataset and baseline results. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 4309–4319

work page 2020

[2] [2]

Md Rabiul Awal, Rui Cao, Roy Ka-Wei Lee, and Sandra Mitrović. 2021. Angry- bert: Joint learning target and emotion for hate speech detection. In Pacific-Asia conference on knowledge discovery and data mining . Springer, 701–713

work page 2021

[3] [3]

Md Rabiul Awal, Roy Ka-Wei Lee, Eshaan Tanwar, Tanmay Garg, and Tanmoy Chakraborty. 2023. Model-agnostic meta-learning for multilingual hate speech detection. IEEE Transactions on Computational Social Systems 11, 1 (2023), 1086– 1095

work page 2023

[4] [4]

Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is space-time atten- tion all you need for video understanding?. In Proceedings of the International Conference on Machine Learning (ICML) , Vol. 2. 4

work page 2021

[5] [5]

R. Cao, M. S. Hee, A. Kuek, W. H. Chong, R. K. W. Lee, and J. Jiang. 2023. Pro- cap: Leveraging a frozen vision-language model for hateful meme detection. In Proceedings of the 31st ACM International Conference on Multimedia . 5244–5252

work page 2023

[6] [6]

Rui Cao and Roy Ka-Wei Lee. 2020. Hategan: Adversarial generative-based data augmentation for hate speech detection. In Proceedings of the 28th International Conference on Computational Linguistics . 6327–6338

work page 2020

[7] [7]

R. Cao, R. K. W. Lee, W. H. Chong, and J. Jiang. 2023. Prompting for Multimodal Hateful Meme Classification. arXiv preprint arXiv:2302.04156 (2023)

arXiv 2023

[8] [8]

Rui Cao, Roy Ka-Wei Lee, and Tuan-Anh Hoang. 2020. DeepHate: Hate speech detection via multi-faceted text representations. In Proceedings of the 12th ACM Conference on Web Science. 11–20

work page 2020