REVIEW 15 cited by
Guiding Instruction-based Image Editing via Multimodal Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Instruction-based image editing improves the controllability and flexibility of image manipulation via natural commands without elaborate descriptions or regional masks. However, human instructions are sometimes too brief for current methods to capture and follow. Multimodal large language models (MLLMs) show promising capabilities in cross-modal understanding and visual-aware response generation via LMs. We investigate how MLLMs facilitate edit instructions and present MLLM-Guided Image Editing (MGIE). MGIE learns to derive expressive instructions and provides explicit guidance. The editing model jointly captures this visual imagination and performs manipulation through end-to-end training. We evaluate various aspects of Photoshop-style modification, global photo optimization, and local editing. Extensive experimental results demonstrate that expressive instructions are crucial to instruction-based image editing, and our MGIE can lead to a notable improvement in automatic metrics and human evaluation while maintaining competitive inference efficiency.
Forward citations
Cited by 15 Pith papers
-
B-repLer: Language-guided Editing of CAD Models
B-repLer fine-tunes a multimodal LLM to locate edits and trains a transformer to modify a B-rep latent code, achieving 53.4% exact-match success on synthetic text-guided editing.
-
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
An MLLM agent that profiles, routes, and reformulates image-editing queries into better-conditioned multi-step operations consistently improves existing editors on hard cases.
-
Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing
VARIN uses a Location-aware Argmax Inversion pseudo-inverse of Gumbel-max sampling to extract editable discrete noises, enabling training-free prompt-guided editing for visual autoregressive models.
-
After the Party: Navigating the Mapping From Color to Ambient Lighting
A new paired dataset and Retinex-based network, RLN2, for restoring images captured under multiple colored light sources to ambient-normalized versions.
-
Beyond Simple Edits: X-Planner for Complex Instruction-Based Image Editing
X-Planner, an MLLM-based planner, decomposes complex image-editing instructions into localized sub-edits with masks and boxes, improving editing quality on standard and new complex benchmarks.
-
Balancing Preservation and Modification: A Region and Semantic Aware Metric for Instruction-Based Image Editing
A region and semantic aware metric for instruction-based image editing, built from LLM parsing plus detection, segmentation, and CLIP directional similarity, reports the highest human alignment among compared metrics.
-
ORIDa: Object-centric Real-world Image Composition Dataset
ORIDa is a public real-world dataset of 200 objects in 30,000+ images with multiple positions per scene, designed for object compositing training and evaluation.
-
Image Editing As Programs with Diffusion Models
IEAP decomposes complex editing instructions into atomic operations executed sequentially on a diffusion transformer, and reports state-of-the-art results on MagicBrush and AnyEdit.
-
VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models
VideoREPA adds a token-relation distillation loss that aligns a text-to-video diffusion model's internal features with VideoMAEv2, boosting physical commonsense scores on VideoPhy and VideoPhy2.
-
Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models
ATLAS adds a Think–Plan–Paint loop with shared positional tokens to unified MLLMs, plus RL-based layout alignment, achieving large reported gains over prior layout-based unified models on compositional image generatio...
-
Instant Preference Alignment for Text-to-Image Diffusion Models
An MLLM-driven, training-free pipeline extracts preference keywords from a reference image and modulates diffusion cross-attention at global and regional levels for instant, multi-round preference-aligned image generation.
-
ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions
A released 6.4 million pair dataset and 613 sample benchmark for instruction-guided image editing of non-rigid motions, plus a Flux.1-dev based baseline that outperforms open-source methods on the new benchmark.
-
ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
A ComfyUI-based multi-agent system with semantic workflow modules and tree-based local-feedback planning reports near-perfect pass rates on ComfyBench and competitive scores on GenEval and Reason-Edit.
-
MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection
MIND-Edit combines instruction rewriting with MLLM-derived visual embeddings to guide diffusion-based image editing, but the reported numbers only partly support the claim of state-of-the-art performance.
-
R-Genie: Reasoning-Guided Generative Image Editing
R-Genie couples a multimodal LLM with a discrete diffusion model to perform image edits that require commonsense reasoning, and introduces a 1,070-triple benchmark called REditBench.
Discussion (0). Sign in to comment.