REVIEW 8 cited by
Supervised Multimodal Bitransformers for Classifying Images and Text
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Self-supervised bidirectional transformer models such as BERT have led to dramatic improvements in a wide variety of textual classification tasks. The modern digital world is increasingly multimodal, however, and textual information is often accompanied by other modalities such as images. We introduce a supervised multimodal bitransformer model that fuses information from text and image encoders, and obtain state-of-the-art performance on various multimodal classification benchmark tasks, outperforming strong baselines, including on hard test sets specifically designed to measure multimodal performance.
Forward citations
Cited by 8 Pith papers
-
EPIC: Efficient Prompt Interaction for Text-Image Classification
EPIC uses similarity-gated cross-modal prompt interactions in frozen CLIP to improve text-image classification accuracy while training about 1% of parameters.
-
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection
MIND uses unlabeled similar memes, bidirectional AI insight derivation, and multi-agent debate to improve zero-shot harmful meme detection on HarM, FHM, and MAMI.
-
AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness
AdamMeme is an adaptive, agent-based evaluation framework that iteratively refines meme text to expose model-specific weaknesses in multimodal models' understanding of meme harmfulness.
-
Class Similarity-Based Multimodal Classification under Heterogeneous Category Sets
The paper introduces MMHCL, a setting where modalities train on heterogeneous category sets, and CSCF, a model using semantic alignment, uncertainty-based dominance selection, and class-similarity fusion to recognize ...
-
MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models
A new 1,565-item benchmark shows large vision-language models classify meme-context relationships far better than they infer the poster's intent.
-
EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detection
EVL-MCoT combines multiple hateful and benign chain-of-thought explanations with prototype-guided vision-language fusion, reporting state-of-the-art harmful meme detection accuracy on HatefulMemes and MultiOFF.
-
Applying multimodal learning to Classify transient Detections Early (AppleCiDEr) I: Data set, methods, and infrastructure
AppleCiDEr combines photometry, images, metadata, and spectra in one deep learning pipeline to classify ZTF transients and variable stars, with high accuracy on common classes but poor performance on tidal disruption events.
-
Co-AttenDWG: Co-Attentive Dimension-Wise Gating and Expert Fusion for Multi-Modal Offensive Content Detection
A new multimodal fusion architecture reports small state-of-the-art gains on two offensive content benchmarks using co-attention, dimension-wise gating, and expert fusion.
Discussion (0). Continue with ORCID to comment.