REVIEW 17 cited by
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Current image generation and editing methods primarily process textual prompts as direct inputs without reasoning about visual composition and explicit operations. We present Generation Chain-of-Thought (GoT), a novel paradigm that enables generation and editing through an explicit language reasoning process before outputting images. This approach transforms conventional text-to-image generation and editing into a reasoning-guided framework that analyzes semantic relationships and spatial arrangements. We define the formulation of GoT and construct large-scale GoT datasets containing over 9M samples with detailed reasoning chains capturing semantic-spatial relationships. To leverage the advantages of GoT, we implement a unified framework that integrates Qwen2.5-VL for reasoning chain generation with an end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance Module. Experiments show our GoT framework achieves excellent performance on both generation and editing tasks, with significant improvements over baselines. Additionally, our approach enables interactive visual generation, allowing users to explicitly modify reasoning steps for precise image adjustments. GoT pioneers a new direction for reasoning-driven visual generation and editing, producing images that better align with human intent. To facilitate future research, we make our datasets, code, and pretrained models publicly available at https://github.com/rongyaofang/GoT.
Forward citations
Cited by 17 Pith papers
-
Demystifying Video Reasoning
Video diffusion models reason along the denoising trajectory (Chain-of-Steps), not primarily across frames, and this mechanism can be nudged by ensembling latent trajectories.
-
FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark
The authors build a 6M-image, 20M-caption reasoning dataset with generation chain-of-thought and a 7-track VLM-judged benchmark, then rank 19 text-to-image models.
-
Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning
Early Generation Intent and Visual Fidelity signals let AdaViG abort low-utility intermediate image generations in UMM math CoT, improving accuracy up to 5.7% and cutting visual FLOPs 25–91%.
-
IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation
Interleaving motion generation with text-motion assessment and refinement improves alignment between generated human motion and goal text.
-
Reconstruction Alignment Improves Unified Multimodal Models
RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...
-
Interleaving Reasoning for Better Text-to-Image Generation
A text-image-text-image multi-turn pipeline improves text-to-image generation, gaining up to 8 points over the base model on several benchmarks while adding an image-conditioned reflection step.
-
ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies
Introduces a benchmark for chain-dependent image editing instructions plus a region-aware consistency metric, and shows a chain-of-thought prompt improves a Gemini-based editor.
-
Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics
Using an inverse dynamics model to label and verify frame transitions lets vision-language models learn forward dynamics and outperform specialized image editors on Aurora-Bench.
-
Beyond Editing Pairs: Fine-Grained Instructional Image Editing via Multi-Scale Learnable Regions
By learning edit regions from CLIP alignment on text-image pairs, the method performs instruction-driven editing without paired editing data, achieving competitive benchmark scores.
-
KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models
A new benchmark, KRIS-Bench, evaluates image editing models on knowledge-grounded reasoning across factual, conceptual, and procedural tasks, and finds large performance gaps in current models.
-
UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation
UniGen shows a 1.5B model trained on open data can beat larger systems on image understanding and generation once it verifies its own outputs with chain-of-thought and Best-of-N selection.
-
SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
A unified multimodal model can improve its own text-to-image generation by using its understanding module as a rewarder in a global-plus-local reward-weighted training loop.
-
Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation
A bar-level symbolic-score song generator (BACH) is claimed to beat published systems and commercial Suno on human-rated quality, duration, and efficiency, but the supporting full text is corrupted and unverifiable.
-
Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation
A new benchmark and agent framework for complex text-to-image generation, with an unvalidated AI-judge evaluation and claims that the agent outperforms GPT-4o on the authors' own benchmark.
-
ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
A ComfyUI-based multi-agent system with semantic workflow modules and tree-based local-feedback planning reports near-perfect pass rates on ComfyBench and competitive scores on GenEval and Reason-Edit.
-
MEPG:Multi-Expert Planning and Generation for Compositionally-Rich Image Generation
MEPG combines LLM-based spatial planning with a mixture of SDXL experts and staged local/global denoising, but the reported gains are modest and the main quality/diversity claims are not backed by experiments.
-
R-Genie: Reasoning-Guided Generative Image Editing
R-Genie couples a multimodal LLM with a discrete diffusion model to perform image edits that require commonsense reasoning, and introduces a 1,070-triple benchmark called REditBench.
Discussion (0). Continue with ORCID to comment.