REVIEW 6 cited by
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks. This paper presents mPLUG, a new vision-language foundation model for both cross-modal understanding and generation. Most existing pre-trained models suffer from the problems of low computational efficiency and information asymmetry brought by the long visual sequence in cross-modal alignment. To address these problems, mPLUG introduces an effective and efficient vision-language architecture with novel cross-modal skip-connections, which creates inter-layer shortcuts that skip a certain number of layers for time-consuming full self-attention on the vision side. mPLUG is pre-trained end-to-end on large-scale image-text pairs with both discriminative and generative objectives. It achieves state-of-the-art results on a wide range of vision-language downstream tasks, such as image captioning, image-text retrieval, visual grounding and visual question answering. mPLUG also demonstrates strong zero-shot transferability when directly transferred to multiple video-language tasks.
Forward citations
Cited by 6 Pith papers
-
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Lavida-O introduces an elastic mixture-of-transformers architecture that brings high-resolution text-to-image generation, object grounding, and image editing into a single masked diffusion model, using planning and se...
-
INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling
INTER is a training-free logit-correction method that adds Harsanyi interaction scores to selected keyword tokens, lowering hallucination on six LVLM benchmarks.
-
GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning
A multi-agent vision-language framework that extracts event, time, and location from public event images, evaluated with a new soft metric on VLM-augmented datasets.
-
DetailMaster: Can Your Text-to-Image Model Handle Long Prompts?
Introduces DetailMaster, a 4,116-prompt benchmark with fine-grained evaluation of long-prompt text-to-image generation, finding that state-of-the-art models achieve only about 50% accuracy on attribute binding and spa...
-
Multimodal Feature Fusion Network with Text Difference Enhancement for Remote Sensing Change Detection
MMChange fuses image features with VLM-generated text descriptions of bitemporal remote sensing images, reporting state-of-the-art IoU/F1 on LEVIR-CD, WHU-CD, and SYSU-CD.
-
ReFrame: Rectification Framework for Image Explaining Architectures
ReFrame wraps image captioning, VQA, and GPT-4 with a Mask R-CNN rectifier, reporting big gains on metrics defined against that same detector.
Discussion (0). Continue with ORCID to comment.