REVIEW 7 cited by
A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Advancing image inpainting is challenging as it requires filling user-specified regions for various intents, such as background filling and object synthesis. Existing approaches focus on either context-aware filling or object synthesis using text descriptions. However, achieving both tasks simultaneously is challenging due to differing training strategies. To overcome this challenge, we introduce PowerPaint, the first high-quality and versatile inpainting model that excels in multiple inpainting tasks. First, we introduce learnable task prompts along with tailored fine-tuning strategies to guide the model's focus on different inpainting targets explicitly. This enables PowerPaint to accomplish various inpainting tasks by utilizing different task prompts, resulting in state-of-the-art performance. Second, we demonstrate the versatility of the task prompt in PowerPaint by showcasing its effectiveness as a negative prompt for object removal. Moreover, we leverage prompt interpolation techniques to enable controllable shape-guided object inpainting, enhancing the model's applicability in shape-guided applications. Finally, we conduct extensive experiments and applications to verify the effectiveness of PowerPaint. We release our codes and models on our project page: https://powerpaint.github.io/.
Forward citations
Cited by 7 Pith papers
-
Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO
A recalibrated GRPO reinforcement learning method lets multimodal LLMs say 'None' for nonexistent referring expressions without sacrificing localization accuracy on objects that do exist.
-
Score-based diffusion models for accurate crystal-structure inpainting and reconstruction of hydrogen positions
Adapting TD-Paint to crystal diffusion models reconstructs hydrogen positions with a LES success rate above 97%, beating unconditioned diffusion and DFT-based inpainting.
-
Beyond Simple Edits: X-Planner for Complex Instruction-Based Image Editing
X-Planner, an MLLM-based planner, decomposes complex image-editing instructions into localized sub-edits with masks and boxes, improving editing quality on standard and new complex benchmarks.
-
DreamLight: Towards Harmonious and Consistent Image Relighting
A unified image- and text-based relighting model with direction-biased attention and a wavelet foreground fixer outperforms existing methods on a synthetic relighting benchmark.
-
Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting
A two-step training loss plus a fine-tuned VAE reduces color and structure discontinuities at mask boundaries in diffusion image inpainting and outpainting.
-
DiGA3D: Coarse-to-Fine Diffusional Propagation of Geometry and Appearance for Versatile 3D Inpainting
DiGA3D performs text-guided 3D inpainting (removal, re-texturing, replacement) with a coarse-to-fine diffusion propagation scheme to improve multi-view appearance and geometry consistency.
-
MTADiffusion: Mask Text Alignment Diffusion Model for Object Inpainting
MTADiffusion improves text-guided object inpainting by training on a new 5M-image mask-text dataset with edge prediction and style-consistency losses.
Discussion (0). Sign in to comment.