Pith. sign in

REVIEW 2 major objections 5 minor 64 references

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multimodal AIs recognize what a meme shows but not why it is funny.

desk verdict A useful survey of multimodal humor with a principled taxonomy, but the headline empirical claim about the interpretation bottleneck rests on an uncontrolled comparison and should be read as a qualitative synthesis rather than a quantitative finding. read the letter →

arxiv 2607.19011 v1 pith:RFXZXTSH submitted 2026-07-21 cs.CL cs.AIcs.MM

Tuo Liang , Zhe Hu , Disheng Liu , Jing Li , Yu Yin This is my paper · ORCID
classification cs.CLcs.AIcs.MM
keywords multimodalhumorunderstandingvisuallargelanguagemodelsincongruitybenchmarkevaluationmemeinterpretationandreasoninggeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the hard part of computational humor is not seeing but understanding: current multimodal large language models identify the objects, roles, and genres in memes, cartoons, and comics with near-human accuracy, yet systematically fail to recover the intended meaning, communicative function, and incongruity structure that make those artifacts funny. To support this, the survey organizes the field into three capabilities—recognition, interpretation and reasoning, and generation—and reports a cross-benchmark evaluation of recent models on seven humor benchmarks with human scores. The pattern is consistent: best models trail humans by 6.58 points on moral selection, 14.22 on title selection, 11.70 on caption selection, and 21.49 on meme question answering, while recognition tasks are largely solved. If the claim is right, progress depends less on bigger multimodal encoders and more on explicit reasoning, cultural grounding, and evaluation that tests meaning rather than labels.

What carries the argument

The organizing device is the capability-centric task hierarchy, which separates recognition (detecting or classifying humorous phenomena), interpretation and reasoning (explaining the mechanism, target, or implicit meaning), and generation (producing humor-consistent outputs conditioned on an artifact). The load-bearing evidence is the cross-benchmark evaluation protocol of Section 6, which reuses each benchmark's original prompts, splits, and scoring rules to compare models across twelve task settings and against human baselines taken from the source papers. This design is what lets the survey attribute score differences to capability level rather than to model family or task name.

What would settle it

Take one set of memes or cartoons and pose recognition-style and interpretation-style questions that are matched for input, answer options, and human-rated difficulty, then run each model multiple times with sampling; if the recognition-to-interpretation gap narrows or disappears under matched formats, the paper's central bottleneck conclusion would be a benchmark artifact rather than a model limitation, and if it persists, the claim stands.

Watch

Extended reading notes

Core claim

The central claim, stated in Section 6.1, is that current MLLMs have made considerable progress on visual and categorical recognition but still fall short in recovering the intended meaning, communicative function, and incongruity structure that make visual humor understandable to humans. The evidence is a capability gradient across benchmarks: recognition-oriented tasks (role classification, humor detection, manga element recognition) reach high accuracy, with the strongest model scoring 97.60% on background recognition and 99.21% on panel localization, whereas interpretation-oriented tasks (YesBut-v2, NYCC, MemeQA) show large and consistent gaps from human performance. The paper also reports that no single model dominates: rankings differ by task, and a reasoning-enhanced variant improves dark-humor detection but hurts several interpretation tasks, so humor understanding is not one monolithic ability. The conclusion the authors draw is that alignment captures what co-occurs but not why, making interpretation and reasoning the central unsolved bottleneck.

Load-bearing premise

The comparison assumes that the benchmarks assigned to the recognition and interpretation levels are comparable in difficulty, so the score gaps reflect capability differences rather than differences in prompt format, answer format, or single-sample decoding noise.

Editorial extensions

If this is right

  • If the interpretation bottleneck is real, then recognition accuracy is a misleading success metric: a model that labels a meme funny may still not know what makes it funny or whom it targets.
  • Interpretation-focused models will need mechanism-grounded reasoning (setup, conflict, resolution), external knowledge about cultural and social norms, and evidence-grounded explanation rather than more aggressive multimodal alignment.
  • Benchmark suites should be built from multiple interpretation tasks spanning different formats and cultures, because model rankings fragment across capabilities; an aggregate accuracy score hides which capability is missing.
  • Sequential, multi-panel humor is where the widest gaps appear, so panel-aware architectures with explicit entity co-reference and setup-punchline decomposition are needed, not single-image processing.
  • Evaluation should move toward rubric-based generative assessment and self-consistency checks, for example penalizing a model that rates something funny yet explains it blandly, because discriminative accuracy inflates apparent understanding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper's bottleneck claim holds, then deployed systems using humor understanding—content moderation, social-media analytics, creative assistance—should treat model outputs as candidate interpretations, not decisions, and require human or rubric verification when the stakes are high.
  • A testable extension would be to use generation as a diagnostic: on the same artifacts, compare a model's explanation faithfulness with its multiple-choice accuracy; models with similar accuracy but different explanation quality would reveal whether the gap is benchmark difficulty or genuine interpretive ability.
  • The large gaps on MemeQA and YesBut-v2 title selection may partly reflect training-data skew toward Western, English-centric humor; a per-culture benchmark with annotations of required background knowledge could separate missing knowledge from missing reasoning, and would sharpen the paper's cultural-grounding challenge into a measurable quantity.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper is a survey of multimodal visual humor understanding for large multimodal models (MLLMs). It organizes the literature with a capability hierarchy spanning recognition, interpretation and reasoning, and generation; a two-way distinction between static visual-textual artifacts (StaVT) and sequential visual narratives (SeqVN); a synthesis of modeling paradigms (cross-modal alignment, grounded reasoning, controllable generation); and a catalogue of benchmarks and evaluation protocols. Beyond the qualitative synthesis, Section 6 contributes a cross-benchmark empirical evaluation of eleven MLLMs on twelve task settings (Table 3), and Section 7 identifies the main barriers to progress: shortcut-prone evaluation, limited cultural and narrative coverage, weak evidence grounding, and unresolved safety and ownership concerns. The central claim, stated in Section 6.1, is that current MLLMs have made considerable progress on visual and categorical recognition but still fall short in recovering the intended meaning, communicative function, and incongruity structure that make visual humor understandable to humans.

Significance. If the central claim holds, the survey provides a useful organizing framework and an up-to-date snapshot of the field; the capability hierarchy and the distinction between descriptive explanation and mechanism-grounded reasoning are genuinely clarifying, and the critique of MCQ and binary evaluation formats is well supported by the reviewed literature. The empirical section is a value-added contribution over a conventional survey: it evaluates a broad and recent model set under a disclosed protocol, and the paper deserves explicit credit for publishing its caveats (prompts differ by design, single-sample decoding, descriptive rather than controlled comparison, stated in Section 6 and Appendix A) and for giving reproducibility details such as the fixed GPT-4o API version and completion date. The bottleneck conclusion is not merely a self-citation artifact: it is independently supported by non-author benchmarks (NYCC, MemeQA) and by the surveyed literature. The main weakness is that the empirical support for the recognition-versus-interpretation contrast is weaker than the Section 6.1 prose suggests, for reasons detailed in the major comments.

major comments (2)
  1. [§6.1, Table 3] The recognition-versus-interpretation contrast is confounded by the composition of the recognition group. Three of the five MangaUB subtasks (RecBg, CharCnt, PanelLoc) are general visual-perception tasks with no humor component, so GPT-4o's 97.60-99.21% scores on them demonstrate perception rather than humor recognition, and including them inflates the recognition column. When the comparison is restricted to humor-specific tasks, the ordering is no longer clean: DarkHumor, the only humor-specific recognition task, tops out at 66.43%, below the best YesBut-v2 Moral score of 84.72%, and HumorDB, also a recognition task, shows a 10.29-point human gap that is larger than the 6.58-point gap on YesBut-v2 Moral. The conclusion that models show considerable progress on visual and categorical recognition should be re-derived on a humor-specific subset, or reframed as a claim about perception rather than about humor recognition.
  2. [§6, Appendix A] The headline gap figures (6.58, 14.22, 11.70, and 21.49 points) are point estimates from one sampled generation per question with do_sample=true, with no variance reporting or significance testing. The disclaimer in Appendix A is correct, but Section 6.1 does not carry its uncertainty into the prose: statements such as 'Interpretation and reasoning remain the central bottleneck' and 'recognition remains challenging when it depends on implicit social norms' are categorical. This matters because several adjacent results in Table 3 differ by amounts larger than the smallest headline gap (the Qwen3-VL-8B versus Qwen3-VL-8B-Thinking comparison alone swings by 4 to 16 points across tasks), so the reader cannot tell which of the reported differences are stable. The paper should report multiple samples or bootstrap confidence intervals for the key within-benchmark contrasts, or explicitly downgrade the Section 6.1 conclusions to descriptive observations.
minor comments (5)
  1. [Table 3] As rendered, the Human row is ambiguous: the values 91.30, 97.50, 94.00, and 81.90 are used in Section 6.1 for YesBut-v2 Moral, YesBut-v2 Title, NYCC, and MemeQA, but in the table layout they appear under the MangaUB subtask columns, and the MangaUB human scores are then missing or double-counted; please align the row or add a per-benchmark footnote. Also, the column header 'Y esBut-v2' contains a spurious space.
  2. [§7.3] The opening sentence claims that 'our cross-benchmark analysis (Section 6) shows that sequential, multi-panel humor exposes the widest model-human gap on panel-sequencing and temporal reordering', but Table 3 contains no panel-sequencing or temporal-reordering task (MangaUB NextInf is next-panel inference and is grouped under recognition), so the sentence should cite the surveyed sequential-humor literature instead of the paper's own analysis.
  3. [Figure 2] The annotation line 'qCaption q Text in Image q Questions q Object bounding boxes' renders as a garbled notation; if this is intended as a bulleted list, the formatting should be fixed in the figure source.
  4. [§6] The paper does not explicitly flag that YesBut-v2 is one of the authors' own benchmarks in the same paragraph where it is used as evidence for the interpretation bottleneck; a one-sentence disclosure would aid the reader, even though the qualitative conclusion is independently supported by NYCC and MemeQA.
  5. [References] Some classic works are cited with reprint years rather than original publication years (for example, the 2024 citation for 'Metaphors We Live By'); please verify that the in-text citations match the intended editions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey's empirical claims rest on external benchmarks; author-built YesBut-v2 is corroborated by independent tasks, and the capability hierarchy is an organizational taxonomy rather than a fitted result.

full rationale

The paper is a survey with an empirical cross-benchmark evaluation; it offers no fitted parameters, no first-principles derivation, and no equation whose output is equivalent to its input by construction. The capability hierarchy in Section 3 is an a priori organizational taxonomy that classifies task types before scores are reported; Section 6 then applies that taxonomy to group benchmark results. The central claim that MLLMs lag on interpretation and reasoning is supported by multiple non-author benchmarks (NYCC, MemeQA, DarkHumor, HumorDB) in addition to the authors' YesBut-v2, so the self-citation is not load-bearing. The paper explicitly cautions that benchmarks use different prompts by design and that results support descriptive within-benchmark comparison rather than a strictly controlled cross-benchmark difficulty analysis, and Appendix A notes that single-sample decoding makes small differences not statistically significant. These are important validity caveats about the strength of the empirical conclusions, but they are not circular reductions: the observed scores are external measurements, not consequences of the survey's definitions. No prediction is fit from a subset of the data, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation. The self-citations are present but peripheral, since the same qualitative pattern appears on independently authored benchmarks.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

The paper is a survey; it introduces no fitted parameters. Its central empirical claims rest on domain assumptions about how to carve up the field, and on the adequacy of the zero-shot evaluation protocol described in Appendix A. The StaVT/SeqVN and three-level capability taxonomies are organizational constructs, not independently verified entities.

assumptions (4)
  • domain assumption Multimodal humor understanding is defined by image-text artifacts whose meaning depends on non-literal mechanisms and shared cultural knowledge.
    Scope-setting definition in §1 and §2.1; excludes audio/video humor and shapes all subsequent synthesis.
  • domain assumption The capability hierarchy of recognition, interpretation & reasoning, and generation is the correct organizing axis, and gains in label prediction do not automatically transfer to explanation.
    Introduced in §3; underlies the survey's structure and the cross-benchmark grouping in Table 3.
  • domain assumption Four mechanisms (incongruity, analogy/conceptual mapping, exaggeration/hyperbole, narrative structure) recurrently explain creative meaning construction.
    §2.3; the list is drawn from prior literature and presented as exhaustive for this survey; if incomplete, the interpretation taxonomy mischaracterizes some works.
  • domain assumption Zero-shot evaluation with benchmark-specific prompts and single sampled generations is adequate for descriptive cross-model comparison.
    Appendix A sets this protocol; the paper itself concedes small differences are not statistically significant, undercutting some comparative conclusions in §6.1.
invented entities (2)
  • StaVT vs SeqVN representation-form dichotomy
    purpose: Categorizes multimodal humor inputs into static image-text artifacts vs sequential visual narratives, used to organize datasets and modeling requirements (§2.2, Figures 1-2).
    An organizational taxonomy specific to this survey; no falsifiable prediction or external validation is provided.
  • Three-level capability hierarchy (recognition, interpretation & reasoning, generation)
    purpose: Organizes tasks, benchmarks, and modeling paradigms into progressively demanding capabilities (§3).
    A conceptual framework rather than an empirically validated construct; the paper argues for its usefulness but offers no independent evidence that these levels correspond to distinct model competencies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges." pith.science (2026). https://pith.science/paper/RFXZXTSH

@misc{pith2026260719011,
  author       = {Pith},
  title        = {Pith review of: Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RFXZXTSH}},
  note         = {Machine review of arXiv:2607.19011}
}
read the original abstract

Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communicative intent rather than literal scene description. This survey focuses on visual humor understanding in single-image and multi-panel artifacts, while treating humor generation as an emerging downstream frontier. We position the literature against prior humor, sarcasm, and general MLLM surveys and organize it using a capability-centric hierarchy spanning recognition, interpretation and reasoning, and generation. Under this lens, we synthesize benchmark design, evaluation protocols, and modeling paradigms, tracing the field's shift from task-specific fusion models to large-model approaches based on multimodal alignment, evidence-grounded reasoning, and controlled generation. We conclude by highlighting the main barriers to progress: shortcut-prone evaluation, limited cultural and narrative coverage, weak evidence grounding, and unresolved safety and ownership concerns.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 35 canonical work pages

  1. [2]

    Dataset Venue Data Forms Mechanism Size Avail

    The Availability column provides links to publicly accessible datasets, and "N/A" indicates unpublished datasets. Dataset Venue Data Forms Mechanism Size Avail. StaVT: Meme Goal: Humor & Entertainment, Satire & Social Critique D-HUMORKasu et al. (2025) ICDM’25 StaVT: Meme Multi 4,379 Link MemeMindGu et al. (2025) Arxiv’25 StaVT: Meme Multi 43,223 N/A TOXI...

    work page 2025

  2. [3]

    StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos

    URL https://api.semanticscholar.org/CorpusID:227227964. Valentin Barriere, Nahuel Gomez, Leo Hemamou, Sofia Callejas, and Brian Ravenet. Standup4ai: A new multilingual dataset for humor detection in stand-up comedy videos.arXiv preprint arXiv:2505.18903,

    work page Pith review arXiv

  3. [6]

    Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!

    Jiwan Chung, Seungwon Lim, Jaehyun Jeon, Seungbeen Lee, and Youngjae Yu. Can visual language models resolve textual ambiguity with visual cues? let visual puns tell you!arXiv preprint arXiv:2410.01023,

    work page Pith review arXiv

  4. [8]

    A survey of multimodal sarcasm detection.arXiv preprint arXiv:2410.18882,

    Shafkat Farabi, Tharindu Ranasinghe, Diptesh Kanojia, Yu Kong, and Marcos Zampieri. A survey of multimodal sarcasm detection.arXiv preprint arXiv:2410.18882,

    arXiv

  5. [11]

    Rubrics as rewards: Reinforcement learning beyond verifiable domains.arXiv preprint arXiv:2507.17746,

    Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains.arXiv preprint arXiv:2507.17746,

    arXiv

  6. [14]

    Decoding the underlying meaning of multimodal hateful memes

    Ming Shan Hee, Wen-Haw Chong, and Ka-Wei Roy Lee. Decoding the underlying meaning of multimodal hateful memes. In32nd International Joint Conference on Artificial Intelligence (IJCAI 2023). International Joint Conferences on Artifical Intelligence (IJCAI),

    work page 2023

  7. [15]

    understanding

    Jack Hessel, Ana Marasovic, Jena D. Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, and Yejin Choi. Do androids laugh at electric sheep? humor “understanding” benchmarks from the new yorker caption contest. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.),Proceedings of the 61st Annual Meeting of the Association for Computational ...

    doi:10.18653/v1/2023

  8. [16]

    Language models, agent models, and world models: The law for machine reasoning and planning.arXiv preprint arXiv:2312.05230,

    Zhiting Hu and Tianmin Shu. Language models, agent models, and world models: The law for machine reasoning and planning.arXiv preprint arXiv:2312.05230,

    arXiv

Show all 64 references
  1. [17]

    Memecap: A dataset for captioning and interpreting memes.arXiv preprint arXiv:2305.13703,

    EunJeong Hwang and Vered Shwartz. Memecap: A dataset for captioning and interpreting memes.arXiv preprint arXiv:2305.13703,

    arXiv

  2. [18]

    Bottlehumor: Self-informed humor explanation using the information bottleneck principle.arXiv preprint arXiv:2502.18331,

    EunJeong Hwang, Peter West, and Vered Shwartz. Bottlehumor: Self-informed humor explanation using the information bottleneck principle.arXiv preprint arXiv:2502.18331,

  3. [19]

    Memeguard: An llm and vlm-based framework for advancing content moderation via meme intervention.arXiv preprint arXiv:2406.05344, 2024a

    Prince Jha, Raghav Jain, Konika Mandal, Aman Chadha, Sriparna Saha, and Pushpak Bhattacharyya. Memeguard: An llm and vlm-based framework for advancing content moderation via meme intervention.arXiv preprint arXiv:2406.05344, 2024a. Prince Jha, Krishanu Maity, Raghav Jain, Apoo...

    arXiv

  4. [20]

    D-humor: Dark humor understanding via multimodal open-ended reasoning–a benchmark dataset and method.arXiv preprint arXiv:2509.06771,

    Sai Kartheek Reddy Kasu, Mohammad Zia Ur Rehman, Shahid Shafi Dar, Rishi Bharat Junghare, Dhanvin Sanjay Namboodiri, and Nagendra Kumar. D-humor: Dark humor understanding via multimodal open-ended reasoning–a benchmark dataset and method.arXiv preprint arXiv:2509.06771,

  5. [21]

    Hope ‘the paragraph guy’explains the rest: Introducing mesum, the meme summarizer

    Anas Anwarul Haq Khan, Tanik Saikh, Arpan Phukan, and Asif Ekbal. Hope ‘the paragraph guy’explains the rest: Introducing mesum, the meme summarizer. InFindings of the Association for Computational Linguistics: EMNLP 2024, pp. 6654–6668,

    2024

  6. [23]

    Looking beyond the pixels: Evaluating visual metaphor understanding in vlms

    17 Manishit Kundu, Sumit Shekhar, and Pushpak Bhattacharyya. Looking beyond the pixels: Evaluating visual metaphor understanding in vlms. InFindings of the Association for Computational Linguistics: EMNLP 2025, pp. 23137–23158,

    2025

  7. [24]

    Are vision-language models safe in the wild? a meme-based benchmark study.arXiv preprint arXiv:2505.15389,

    DongGeon Lee, Joonwon Jang, Jihae Jeong, and Hwanjo Yu. Are vision-language models safe in the wild? a meme-based benchmark study.arXiv preprint arXiv:2505.15389,

  8. [26]

    Beneath the surface: Unveiling harmful memes with multimodal reasoning distilled from large language models.arXiv preprint arXiv:2312.05434,

    Hongzhan Lin, Ziyang Luo, Jing Ma, and Long Chen. Beneath the surface: Unveiling harmful memes with multimodal reasoning distilled from large language models.arXiv preprint arXiv:2312.05434,

    arXiv

  9. [27]

    Towards multi-modal sarcasm detection via hierarchical congruity modeling with knowledge enhancement.arXiv preprint arXiv:2210.03501,

    Hui Liu, Wenya Wang, and Haoliang Li. Towards multi-modal sarcasm detection via hierarchical congruity modeling with knowledge enhancement.arXiv preprint arXiv:2210.03501,

    arXiv

  10. [28]

    G-eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634,

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634,

    arXiv

  11. [29]

    Inference-time scaling for generalist reward modeling.arXiv preprint arXiv:2504.02495,

    Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling.arXiv preprint arXiv:2504.02495,

  12. [30]

    Comicorda: Dialogue act recognition in comic books

    Jiří Martínek, Pavel Král, Ladislav Lenc, and Josef Baloun. Comicorda: Dialogue act recognition in comic books. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 3566–3578,

    2024

  13. [31]

    Yesbut: A high-quality annotated multimodal dataset for evaluating satire comprehension capability of vision-language models.arXiv preprint arXiv:2409.13592,

    Abhilash Nandy, Yash Agarwal, Ashish Patwa, Millon Madhur Das, Aman Bansal, Ankit Raj, Pawan Goyal, and Niloy Ganguly. Yesbut: A high-quality annotated multimodal dataset for evaluating satire comprehension capability of vision-language models.arXiv preprint arXiv:2409.13592,

    arXiv

  14. [32]

    Which llms get the joke? probing non-stem reasoning abilities with humorbench.arXiv preprint arXiv:2507.21476,

    Reuben Narad, Siddharth Suresh, Jiayi Chen, Pine SL Dysart-Bricken, Bob Mankoff, Robert Nowak, Jifan Zhang, and Lalit Jain. Which llms get the joke? probing non-stem reasoning abilities with humorbench.arXiv preprint arXiv:2507.21476,

    arXiv

  15. [33]

    Benchmarking vision language models for cultural understanding

    18 Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd Van Steenkiste, Lisa Anne Hendricks, Karolina Stańczak, and Aishwarya Agrawal. Benchmarking vision language models for cultural understanding. InProceedings of the 2024 Conference on Empirical Methods in Natural L...

    2024

  16. [34]

    Laugh, relate, engage: Stylized comment generation for short videos.arXiv preprint arXiv:2511.03757,

    Xuan Ouyang, Senan Wang, Bouzhou Wang, Siyuan Xiahou, Jinrong Zhou, and Yuekang Li. Laugh, relate, engage: Stylized comment generation for short videos.arXiv preprint arXiv:2511.03757,

  17. [35]

    Yamshchikov

    Sandro Paval, Pascal Meißner, and Ivan P. Yamshchikov. ComicScene154: A scene dataset for comic analysis. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.),Proceedings of the 2025 Conference on Empirical Methods in Natural Language Proces...

    2025

  18. [36]

    ISBN 979-8-89176-332-6

    Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main.1608. URL https://aclanthology.org/2025.emnlp-main.1608/. Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. MOMENTA: A m...

    2025 doi

  19. [37]

    Can large language models understand symbolic graphics programs? arXiv preprint arXiv:2408.08313,

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Zhen Liu, Tim Z Xiao, Katherine M Collins, Joshua B Tenenbaum, Adrian Weller, Michael J Black, and Bernhard Schölkopf. Can large language models understand symbolic graphics programs? arXiv preprint arXiv:2408.08313,

    arXiv

  20. [39]

    Understanding figurative meaning through explainable visual entailment

    Arkadiy Saakyan, Shreyas Kulkarni, Tuhin Chakrabarty, and Smaranda Muresan. Understanding figurative meaning through explainable visual entailment. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hum...

    2025

  21. [40]

    Memeclip: Leveraging clip representations for multimodal meme classification.arXiv preprint arXiv:2409.14703,

    Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, and Haohan Wang. Memeclip: Leveraging clip representations for multimodal meme classification.arXiv preprint arXiv:2409.14703,

    arXiv

  22. [42]

    Semeval-2020 task 8: Memotion analysis–the visuo-lingual metaphor!arXiv preprint arXiv:2008.03781,

    Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas Pykl, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Bjorn Gamback. Semeval-2020 task 8: Memotion analysis–the visuo-lingual metaphor!arXiv preprint arXiv:2008.03781,

    2020 arXiv

  23. [43]

    Disarm: Detecting the victims targeted by harmful memes.arXiv preprint arXiv:2205.05738, 2022a

    19 Shivam Sharma, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. Disarm: Detecting the victims targeted by harmful memes.arXiv preprint arXiv:2205.05738, 2022a. Shivam Sharma, Firoj Alam, Md Shad Akhtar, Dimitar Dimitrov, Giovanni Da San Martino, Hamed Firooz, Alon Hal...

    arXiv

  24. [45]

    doi: 10.18653/v1/2024.findings-naacl.152

    Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-naacl.152. URL https://aclanthology.org/2024.findings-naacl.152/. Binghao Tang, Boda Lin, Haolong Yan, and Si Li. Leveraging generative large language models with visual instruction and demonstration ret...

    2024 doi

  25. [46]

    Humor mechanics: Advancing humor generation with multistep reasoning

    Alexey Tikhonov and Pavel Shtykovskiy. Humor mechanics: Advancing humor generation with multistep reasoning. arXiv preprint arXiv:2405.07280,

    arXiv

  26. [47]

    Memecraft: Contextual and stance-driven multimodal meme generation

    Han Wang and Roy Ka-Wei Lee. Memecraft: Contextual and stance-driven multimodal meme generation. InProceedings of the ACM Web Conference 2024, pp. 4642–4652,

    2024

  27. [48]

    Innovative thinking, infinite humor: Humor research of large language models through structured thought leaps.arXiv preprint arXiv:2410.10370, 2024a

    Han Wang, Yilin Zhao, Dian Li, Xiaohan Wang, Gang Liu, Xuguang Lan, and Hui Wang. Innovative thinking, infinite humor: Humor research of large language models through structured thought leaps.arXiv preprint arXiv:2410.10370, 2024a. Jiquan Wang, Lin Sun, Yi Liu, Meizhi Shao, an...

    arXiv

  28. [49]

    Xiaochen Wang, Heming Xia, Jialin Song, Longyu Guan, Qingxiu Dong, Rui Li, Yixin Yang, Yifan Pu, Weiyao Luo, Yiru Wang, Xiangdi Meng, Wenjie Li, and Zhifang Sui. Beyond single frames: Can LMMs comprehend implicit narratives in comic strip? In Christos Christodoulopoulos, Tanmo...

    2025

  29. [50]

    ISBN 979-8-89176-335-7

    Association for Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.342. URLhttps://aclanthology.org/2025.findings-emnlp.342/. Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Fuxiao Liu, Ged...

    2025 doi

  30. [51]

    Taxonomy of risks posed by language models

    20 Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al. Taxonomy of risks posed by language models. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and...

    2022

  31. [52]

    Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models.arXiv preprint arXiv:2504.15279,

    Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models.arXiv preprint arXiv:2504.15279,

    arXiv

  32. [53]

    Yixin Yang, Zheng Li, Qingxiu Dong, Heming Xia, and Zhifang Sui. Can large multimodal models uncover deep semantics behind images? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.),Findings of the Association for Computational Linguistics: ACL 2024, pp. 1898–1912, Bangko...

    2024

  33. [54]

    doi: 10.18653/v1/2024.findings-acl.113

    Association for Compu- tational Linguistics. doi: 10.18653/v1/2024.findings-acl.113. URLhttps://aclanthology.org/2024.findings-acl.113/. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models.National Scienc...

    2024 doi

  34. [55]

    Mmoe: Enhancing multimodal models with mixtures of multimodal interaction experts

    Haofei Yu, Zhengyang Qi, Lawrence Keunho Jang, Russ Salakhutdinov, Louis-Philippe Morency, and Paul Pu Liang. Mmoe: Enhancing multimodal models with mixtures of multimodal interaction experts. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Proces...

    2024

  35. [56]

    Image matters: A new dataset and empirical study for multimodal hyperbole detection

    Huixuan Zhang and Xiaojun Wan. Image matters: A new dataset and empirical study for multimodal hyperbole detection. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 8652–8661,

    2024

  36. [57]

    Humorchain: Theory-guided multi-stage reasoning for interpretable multimodal humor generation.arXiv preprint arXiv:2511.21732,

    Jiajun Zhang, Shijia Luo, Ruikang Zhang, and Qi Su. Humorchain: Theory-guided multi-stage reasoning for interpretable multimodal humor generation.arXiv preprint arXiv:2511.21732,

  37. [58]

    Humor in ai: Massive scale crowd-sourced preferences and benchmarks for cartoon captioning.Advances in Neural Information Processing Systems, 37:125264–125286, 2024a

    Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy T Rogers, Kevin G Jamieson, et al. Humor in ai: Massive scale crowd-sourced preferences and benchmarks for cartoon captioning.Advances in Neural Information Pr...

    1904 arXiv

  38. [59]

    Somelvlm: A large vision language model for social media processing

    Xinnong Zhang, Haoyu Kuang, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuan-Jing Huang, and Zhongyu Wei. Somelvlm: A large vision language model for social media processing. InFindings of the Association for Computational Linguistics: ACL 2024, pp. 2366–2389, 2024b...

    2024 arXiv

  39. [60]

    Social meme-ing: Measuring linguistic variation in memes

    Naitian Zhou, David Jurgens, and David Bamman. Social meme-ing: Measuring linguistic variation in memes. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p...

    2024

  40. [61]

    For each benchmark, we retain the task definition, prompt format, answer format, evaluation split, and scoring procedure reported in the corresponding original paper

    21 A Evaluation Details We evaluate all models in a zero-shot setting. For each benchmark, we retain the task definition, prompt format, answer format, evaluation split, and scoring procedure reported in the corresponding original paper. Except for GPT-4o, all evaluated models...

    2024

  41. [62]

    For open-source models, decoding is performed withdo_sam- ple=true; all remaining benchmark-specific generation and evaluation settings follow the corresponding original papers

    Each model is queried once per question. For open-source models, decoding is performed withdo_sam- ple=true; all remaining benchmark-specific generation and evaluation settings follow the corresponding original papers. When an official evaluation split is available, we use it ...

    2016

  42. [64]

    Dataset Venue Mechanism Size Avail

    The Availability column provides links to publicly accessible datasets, and "N/A" indicates unpublished datasets. Dataset Venue Mechanism Size Avail. StaVT: Meme|Goal:Humor & Entertainment, Satire & Social Critique MemeReaConZhao et al. (2025) EMNLP’25 Multi 1,565 N/A MEMESAFE...

    2025

  43. [1980]

    Mememind: A large-scale multimodal dataset with chain-of-thought reasoning for harmful meme detection.arXiv preprint arXiv:2506.18919,

    Hexiang Gu, Qifan Yu, Saihui Hou, Zhiqin Fang, Huijia Wu, and Zhaofeng He. Mememind: A large-scale multimodal dataset with chain-of-thought reasoning for harmful meme detection.arXiv preprint arXiv:2506.18919,

  44. [1996]

    Memedetoxnet: Balancing toxicity reduction and context preservation

    Gitanjali Kumari, Jitendra Solanki, and Asif Ekbal. Memedetoxnet: Balancing toxicity reduction and context preservation. InFindings of the Association for Computational Linguistics: ACL 2025, pp. 25076–25098,

    2025

  45. [2004]

    Spoken in jest, detected in earnest: A systematic review of sarcasm recognition-multimodal fusion, challenges, and future prospects.IEEE Transactions on Affective Computing, 2025a

    Xiyuan Gao, Shekhar Nayak, and Matt Coler. Spoken in jest, detected in earnest: A systematic review of sarcasm recognition-multimodal fusion, challenges, and future prospects.IEEE Transactions on Affective Computing, 2025a. Xiyuan Gao, Bruce Xiao Wang, Meiling Zhang, Shuming H...

    2025

  46. [2013]

    Content-specific humorous image captioning using incongruity resolution chain-of-thought

    Kohtaro Tanaka, Kohei Uehara, Lin Gu, Yusuke Mukuta, and Tatsuya Harada. Content-specific humorous image captioning using incongruity resolution chain-of-thought. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.),Findings of the Association for Computational Linguistics: N...

    2024

  47. [2015]

    Vipe: Visualise pretty-much everything.arXiv preprint arXiv:2310.10543,

    Hassan Shahmohammadi, Adhiraj Ghosh, and Hendrik Lensch. Vipe: Visualise pretty-much everything.arXiv preprint arXiv:2310.10543,

    arXiv

  48. [2016]

    Are we on the right way for evaluating large vision-language models?Advances in Neural Information Processing Systems, 37:27056–27087, 2024a

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models?Advances in Neural Information Processing Systems, 37:27056–27087, 2024a. Ruirui Chen...

  49. [2019]

    I spy a metaphor: Large language models and diffusion models co-create visual metaphors

    Tuhin Chakrabarty, Arkadiy Saakyan, Olivia Winn, Artemis Panagopoulou, Yue Yang, Marianna Apidianaki, and Smaranda Muresan. I spy a metaphor: Large language models and diffusion models co-create visual metaphors. arXiv preprint arXiv:2305.14724,

    arXiv

  50. [2020]

    Mememqa: multimodal question answering for memes via rationale-based inferencing.arXiv preprint arXiv:2405.11215,

    Siddhant Agarwal, Shivam Sharma, Preslav Nakov, and Tanmoy Chakraborty. Mememqa: multimodal question answering for memes via rationale-based inferencing.arXiv preprint arXiv:2405.11215,

    arXiv

  51. [2021]

    Textmi: Textualize multimodal information for integrating non-verbal cues in pre-trained language models.arXiv preprint arXiv:2303.15430,

    Md Kamrul Hasan, Md Saiful Islam, Sangwu Lee, Wasifur Rahman, Iftekhar Naim, Mohammed Ibrahim Khan, and Ehsan Hoque. Textmi: Textualize multimodal information for integrating non-verbal cues in pre-trained language models.arXiv preprint arXiv:2303.15430,

    arXiv

  52. [2022]

    When’yes’meets’ but’: Can large models comprehend contradictory humor through comparative reasoning?arXiv preprint arXiv:2503.23137,

    Tuo Liang, Zhe Hu, Jing Li, Hao Zhang, Yiren Lu, Yunlai Zhou, Yiran Qiao, Disheng Liu, Jeirui Peng, Jing Ma, et al. When’yes’meets’ but’: Can large models comprehend contradictory humor through comparative reasoning?arXiv preprint arXiv:2503.23137,

    arXiv

  53. [2023]

    Chumor 1.0: A truly funny and challenging chinese humor understanding dataset from ruo zhi ba.arXiv preprint arXiv:2406.12754,

    Ruiqi He, Yushu He, Longju Bai, Jiarui Liu, Zhenjie Sun, Zenghao Tang, He Wang, Hanchen Xia, and Naihao Deng. Chumor 1.0: A truly funny and challenging chinese humor understanding dataset from ruo zhi ba.arXiv preprint arXiv:2406.12754,

    arXiv

  54. [2024]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261,

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabil...

    arXiv

  55. [2025]

    Ashwin Baluja

    URLhttps://arxiv.org/abs/2511.21631. Ashwin Baluja. Text is not all you need: Multimodal prompting helps llms understand humor. InProceedings of the 1st Workshop on Computational Humor (CHum), pp. 9–17,

    arXiv

  56. [2026]

    Humor in pixels: Benchmarking large multimodal models understanding of online comics

    Yuriel Ryan, Rui Yang Tan, Kenny Tsu Wei Choo, and Roy Ka-Wei Lee. Humor in pixels: Benchmarking large multimodal models understanding of online comics. InFindings of the Association for Computational Linguistics: EMNLP 2025, pp. 14024–14050,

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.