REVIEW 13 cited by
Kosmos-G: Generating Images in Context with Multimodal Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in subject-driven image generation have made significant strides. However, current methods still fall short in diverse application scenarios, as they require test-time tuning and cannot accept interleaved multi-image and text input. These limitations keep them far from the ultimate goal of "image as a foreign language in image generation." This paper presents Kosmos-G, a model that leverages the advanced multimodal perception capabilities of Multimodal Large Language Models (MLLMs) to tackle the aforementioned challenge. Our approach aligns the output space of MLLM with CLIP using the textual modality as an anchor and performs compositional instruction tuning on curated data. Kosmos-G demonstrates an impressive capability of zero-shot subject-driven generation with interleaved multi-image and text input. Notably, the score distillation instruction tuning requires no modifications to the image decoder. This allows for a seamless substitution of CLIP and effortless integration with a myriad of U-Net techniques ranging from fine-grained controls to personalized image decoder variants. We posit Kosmos-G as an initial attempt towards the goal of "image as a foreign language in image generation." The code can be found at https://aka.ms/Kosmos-G
Forward citations
Cited by 13 Pith papers
-
MultiRef: Controllable Image Generation with Multiple Visual References
MultiRef-bench shows that current image generators that accept multiple visual references still fail to combine them reliably, with the best tested model OmniGen reaching only 66.6% synthetic and 79.0% real-world alig...
-
UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
An MLLM-conditioned next-scale VAR decoder handles 15+ unified visual generation tasks with competitive quality and substantially lower latency than diffusion baselines.
-
Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing
RL3DEdit fine-tunes FLUX-Kontext with GRPO using VGGT confidence and pose rewards to produce multi-view consistent 3D scene edits in a single pass.
-
TUNA: Comprehensive Fine-grained Temporal Understanding Evaluation on Dense Dynamic Videos
TUNA introduces a 1,000-video benchmark with dense temporal captions and 1,432 multiple-choice questions, and finds that current video LMMs are weakest at camera motion, action sequences, and multi-subject scenes.
-
Dynamic-I2V: Exploring Image-to-Video Generation Models via Multimodal LLM
An adapter that injects Qwen2VL multimodal features into CogVideoX-I2V improves dynamic range on the authors' new DIVE benchmark, but the SOTA claims rest mainly on that self-designed metric.
-
Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrieval
UNITE combines curated multimodal training data and a modality-masked contrastive loss to achieve strong retrieval performance across text, image, and video tasks.
-
MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models
MMIG-Bench is a unified benchmark of 4,850 prompts and 1,750 reference images with a three-level evaluation suite, including the VQA-based Aspect Matching Score that correlates with human ratings.
-
I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models
ThinkDiff aligns vision-language model features to a T5 decoder via captioning, then injects those features into a T5-based diffusion decoder, achieving 46.3% on the CoBSAT benchmark without reasoning-specific training data.
-
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
MEDIC-AD adds anomaly-aware and difference tokens to a medical VLM, claiming SOTA lesion detection, temporal tracking, and visual grounding; the zero-shot claim is undermined by likely train/test overlap.
-
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.
-
VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
Merging a coding LLM into a vision-language model via task vectors yields an open-source multimodal coder that reaches near-GPT-4o performance on the authors' new benchmark.
-
Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects
A tuning-free framework that jointly preserves multiple reference subject identities and obeys bounding-box layout constraints in text-to-image diffusion.
-
FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy
The abstract claims a new visuotactile robot manipulation policy (FBI) that outperforms baselines, but the manuscript body is an unrelated paper on text-to-image synthesis, so the claimed result is absent.
Discussion (0). Continue with ORCID to comment.