Pith. sign in

REVIEW 5 major objections 5 minor 41 references

FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free tweak to Stable Diffusion inpainting inputs can lift prompt-following by up to 60%.

desk verdict A plausible training-free inpainting plug-in whose headline result is inflated by tuning on the same benchmark used for evaluation. read the letter →

arxiv 2412.00427 v1 pith:6ZQGLIYW submitted 2024-11-30 cs.CV cs.AI

Teng-Fang Hsiao , Bo-Kai Ruan , Sung-Lin Tsai , Yi-Lun Wu , Hong-Han Shuai This is my paper · ORCID
classification cs.CVcs.AI
keywords text-guidedinpaintingStableDiffusiontraining-freecross-attentionclassifier-freeguidanceinputconditioningFCIBenchinstructionfollowing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Stable Diffusion Inpainting (SDI) models fail to follow complex prompts when the surrounding image context is unrelated, because the model was trained with random masking and leans on the image condition to stay harmonious. It locates the mechanism in cross-attention: the mask input shifts query-key features so the model paints inside the mask, and the image condition supplies context that can override the prompt. From this it derives FreeCond, a two-part, training-free modification of the network's input conditions—low-pass filtering the image condition in early diffusion steps and scaling the mask condition—that it claims improves prompt adherence and mask fitting on any SDI-based model with no extra computation. The paper also introduces FCIBench, a 600-pair benchmark with precise, rough, and multiple masks paired with complex prompts unrelated to the image, and reports up to a 60% CLIP-score gain for SDI and 58% for SDXL on it.

What carries the argument

The load-bearing object is the cross-attention layer in the SDI UNet, where query features $Q$ from the concatenated noise, mask, and image latents meet prompt key and value tokens $K, V$. The paper defines a Channel Influence Indicator $\mathrm{CI}(Q, M, k, i)$ to show that the mask condition shifts attention channels, particularly the first 80 channels, toward informative prompt tokens. FreeCond's machinery is the pair of input modulations: Eq. (3) low-pass filters the image latent at early diffusion steps to reduce contextual interference, and Eq. (4) scales the mask latent to intensify the cross-attention feature shift. Together they make the modified input conditions carry the work that would otherwise require retraining.

What would settle it

Take a held-out set of complex inpainting prompts not used in the paper's ablations, freeze FreeCond's hyperparameters at the values reported, and measure the CLIP-score gain over the plain SDI baseline; if the gain drops to the ~1% level observed on COCO and BrushBench rather than the claimed ~60%, the 'free lunch' framing fails. A second check is to measure LPIPS or PSNR on the preserved background with and without the low-pass filter to see whether the assumed background preservation holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that SDI's instruction-following weakness is a conditioning bias, not a capacity problem, and can be corrected by modulating only the inputs to the denoising UNet. Concretely, replacing the image condition with $z_{fc} = \mathrm{LPF}(z_c, \gamma)$ for early timesteps $t \ge t_{fc}$ removes high-frequency context that would otherwise dominate generation, while replacing the mask condition with $M_{fc} = \alpha M_c + \beta(1 - M_c)$ amplifies the cross-attention feature shift that directs prompt influence into the masked region. The resulting noise predictor $\hat{\epsilon}_\theta(z_t, z_{fc}, M_{fc}, t, p)$ is argued to be a more general form of classifier-free guidance. The paper reports consistent gains in CLIP and IoU scores across five SDI-based baselines, with the largest gains on the new hard benchmark, FCIBench, including a 60% CLIP-score increase over the original SDI and a 1% increase over the existing SOTA.

Load-bearing premise

The central claim rests on the assumption that the four FreeCond hyperparameters, chosen by ablations on the same FCIBench benchmark that produces the reported 60% gain, continue to work well on other inpainting prompts; a related assumption is that low-pass filtering the image condition in early steps leaves the background intact enough for preservation metrics to hold.

Editorial extensions

If this is right

  • Any SDI-based inpainting model can be upgraded by wrapping its inputs in FreeCond, with no retraining and no extra compute.
  • The method improves prompt adherence and mask fitting at the same time, rather than trading one for the other as classifier-free guidance scaling does.
  • FCIBench gives the community a harder evaluation setting in which complex prompts are unrelated to the image context, exposing weaknesses that COCO and BrushBench hide.
  • On easy benchmarks such as COCO and BrushBench the gains are small (about 1% or less), so FreeCond's value is concentrated in hard, context-unrelated inpainting requests.
  • FreeCond also improves SOTA methods like BrushNet and PowerPaint by about 1% CLIP on FCIBench, suggesting the plug-in is complementary to training-based mask-fitting solutions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the same input-condition modulation could transfer to other conditional diffusion pipelines, such as depth-to-image or image editing, since the paper's mechanism is about conditioning dominance in cross-attention rather than about inpainting per se; the paper does not test this.
  • I infer that the headline 60% figure should be read as benchmark-specific: if the four hyperparameters are fixed and applied to a held-out collection of complex prompts, the gain may shrink toward the single-digit percentages seen on COCO, because the paper tuned those values on FCIBench itself.
  • A natural testable extension is to make $\alpha$, $\beta$, $\gamma$, and $t_{fc}$ adaptive to prompt complexity or mask size, since the paper reports fixed values and notes that over-adjustment degrades image quality.
  • I infer that the method's real-world niche is interactive editing tools where users give detailed prompts on cluttered backgrounds; for simple object replacement, users should expect no visible change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper analyzes Stable Diffusion Inpainting's instruction-following deficiency, attributing it to training bias from random masking and to the model's reliance on the image condition. It proposes FreeCond, a training-free modification of the input conditions: low-pass filtering the image latent in early diffusion steps (Eq. 3) and scaling the mask condition as M_fc = α M_c + β (1 − M_c) (Eq. 4). The authors argue that this aligns cross-attention features with the model's learned bias and improves prompt-adherence and mask-fitting. They introduce FCIBench and report improved CLIP and IoU across SDI, ControlNet Inpainting, HD-Painter, PowerPaint, BrushNet, and SDXL, including a headline 60% CLIP gain for SDI and 58% for SDXL on FCIBench.

Significance. If the central claim holds, FreeCond would be a useful plug-in for SDI-based inpainting models: it requires no training, no additional computation, and is compatible with existing baselines. The paper also contributes an interpretability analysis based on a channel influence indicator and proposes a benchmark with complex prompts and varied mask types. However, the current evidence is not fully convincing: the headline improvement is measured on the authors' own benchmark after selecting the four FreeCond hyperparameters on that benchmark, the FCIBench baseline is unusually low, and the proposed mask modulation is not compared against simple mask dilation. These issues are addressable, so the work is promising but needs revision.

major comments (5)
  1. [Sec. 5.3, Fig. 10] The four FreeCond hyperparameters α, β, γ, and t_fc are selected through ablations that report CLIP and IoU on the same three benchmarks, including FCIBench, that produce the headline numbers. No held-out split, no pre-registered configuration, and no error bars are provided. The reported 60% CLIP gain for SDI on FCIBench (Tab. 2) is therefore partly a fitted-prediction result. Please report performance with a configuration fixed before evaluation, or use cross-validation, and give variance estimates across the 600 prompts.
  2. [Tab. 2] On FCIBench, the SDI baseline CLIP score is 11.45 and FreeCond raises it to 18.27; on COCO and BrushBench the same baseline changes by only +0.15 and +0.05 CLIP points. The headline 60% relative gain is thus dominated by the self-built benchmark with an unusually low baseline. Please report absolute gains on standard benchmarks and justify why FCIBench is the appropriate basis for the central claim, or moderate the headline claim accordingly.
  3. [Sec. 4.2, Eq. (4); Sec. 3.3] M_fc = α M_c + β (1 − M_c) with β > 0 is a soft dilation or offset of the mask condition. Since Sec. 3.3 already shows that enlarging the mask improves CLIP and IoU in the hard-instruction regime, a baseline that simply dilates the input mask, without the low-pass image modification, is needed. Tab. 2 does not include such a baseline, so the specific contribution of the FreeCond modulation cannot be isolated from the already-known effect of mask dilation.
  4. [Sec. 4.1, Eq. (3); Tab. 2] The claim that low-pass filtering z_fc in early steps largely preserves the background is not well supported by the reported metrics: in Tab. 2, SDI's PSNR drops from 25.95 to 24.79 on COCO, from 27.26 to 26.73 on BrushBench, and from 25.54 to 24.58 on FCIBench when FreeCond is applied. Please quantify background preservation more directly, for example by reporting PSNR and LPIPS on the unmasked region only, and temper the 'minimal impact' claim accordingly.
  5. [Sec. 3.4, Fig. 7] The claim that the first 80 channels of the cross-attention query are specifically adapted to the mask is based on a single visualization of ΔCI for one example. Since Sec. 4.2's design of M_fc relies on this claim, please provide statistics over multiple prompts, masks, and timesteps, or otherwise weaken the architectural claim to an observation that motivated the heuristic.
minor comments (5)
  1. [Abstract] There is a typo: 'SDXLI' should be 'SDXL'.
  2. [Sec. 4.1] There is a typo: 'insturction' should be 'instruction'.
  3. [Eq. (2)] The typesetting of the normalization term in the CI definition is garbled ('1P j ¯Mj H×W /16X j=1'); please rewrite the equation clearly.
  4. [Footnote 1 and Sec. 5.1] The paper says the full version and code are available in an external repository; for a journal submission, the full experimental settings (per-baseline hyperparameters, exact LPF implementation, and IoU computation details) should be included in the paper or a stable supplement so that the reported numbers are verifiable.
  5. [References] Reference [35] is a duplicate of reference [34]; please remove the duplicate and renumber.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 60% FCIBench CLIP gain is obtained after tuning FreeCond's four hyperparameters on FCIBench itself, so the central 'free lunch' claim is partly a fit rather than an independent prediction.

  1. fitted input called prediction [Sec. 5.3 (Ablation Study), Fig. 10; Abstract and Tab. 2]
    "In Fig. 10 and Fig. 11, we examine the impact of adjusting five components: (a) the classifier-free guidance (CFG) scale w [9], (b) the inner-mask scale α, (c) the outer-mask scale β, (d) the LPF threshold γ with a fixed tf c= 25, and (e) the LPF timestep tf cwith γ = 0 .75π. For each test, we fix the parameters at (w, α, β, γ, tf c) = (15 , 1, 0, π, T) (the default configuration of original SDI) and vary only one parameter at a time. Based on quantitative and qualitative outcomes, we summarize our findings below."

    This ablation sweep is reported on COCO, BrushBench, and FCIBench (Fig. 10's rightmost block), so the α, β, γ, and t_fc values used to produce Tab. 2 are selected using FCIBench CLIP/IoU — the same benchmark and metric that generate the headline '60% increase' (SDI CLIP 11.45 → 18.27 in Tab. 2). No held-out split or configuration fixed before evaluation is reported. The central claim 'yielding up to a 60% and 58% improvement of SDI and SDXLI in the CLIP score' therefore reports the best configuration found by optimizing on the evaluation set, i.e., a fitted number presented as a predicted gain. The improvement is not an identity, but the headline result is statistically forced by selection on the very benchmark used for the claim.

full rationale

The main circularity is the fitted-prediction pattern: the four FreeCond hyperparameters (α, β, γ, t_fc) are selected via the Sec. 5.3 / Fig. 10 sweeps, and Fig. 10 evaluates CLIP and IoU on FCIBench, which is exactly the benchmark used for the paper's headline '60% increase in CLIP score of SDI backbone.' Since the same benchmark and metric are used both for selection and for the final headline number, the reported improvement is at least partly a fit rather than an independent prediction. The paper explicitly notes that the full version, including detailed per-baseline hyperparameter settings, is deferred to an external repository ('Due to arXiv file size limitations, we provide an abbreviated version of the paper here; the full version can be accessed in the repository'), so the Tab. 2 configurations cannot be independently checked here. The external-benchmark results (COCO and BrushBench gains near or below 1%) are more honest estimates and show that the method has some independent content. The self-citations in the reference list are not load-bearing for the main derivation, and no definitional equation-level circularity was found. Score 6 is appropriate because the central empirical claim reduces partly to selection on the evaluation benchmark, but not to an exact identity or a pure self-citation chain.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The Channel Influence Indicator is a diagnostic metric, not a postulated entity. The central empirical claim rests on four hand-tuned input-condition parameters and on assumptions about diffusion frequency behavior and the SDI training distribution.

free parameters (4)
  • alpha (inner mask scale) = not specified in arXiv text
    Scaling factor for the masked region in M_fc = alpha * M_c + beta * (1 - M_c), Eq. (4). Tuned via ablations on FCIBench (Sec. 5.3).
  • beta (outer mask scale) = not specified in arXiv text
    Scaling factor for the unmasked region in Eq. (4). Tuned via ablations on FCIBench (Sec. 5.3).
  • gamma (low-pass filter threshold) = not specified in arXiv text
    Cutoff for low-pass filtering of the image condition in Eq. (3). Tuned via ablations on FCIBench (Sec. 5.3).
  • t_fc (timestep threshold) = not specified in arXiv text
    Timestep at which the image condition switches from low-pass filtered to original in Eq. (3). Tuned via ablations on FCIBench (Sec. 5.3).
assumptions (3)
  • domain assumption Low-frequency image content is formed in early diffusion steps and high-frequency details appear later, so low-pass filtering z_c in early steps preserves background while reducing context interference.
    Invoked in Sec. 4.1 Eq. (3), citing [2, 28, 35]. This is a standard observation in diffusion literature, but its validity for the inpainting UNet and the chosen cutoff is assumed, not demonstrated by the paper.
  • ad hoc to paper Certain channels of the cross-attention query, especially the first 80, are specifically adapted to the input mask and drive selective prompt influence.
    This is the paper's own hypothesis from Sec. 3.4, supported by the CI indicator in Eq. (2) and Fig. 7, but not independently verified.
  • ad hoc to paper COCO mask statistics are a valid surrogate for the real SDI training mask distribution.
    Stated in Sec. 3.2: 'the exact SDI training mask distribution is not accessible, our analysis on the COCO dataset as a surrogate'. The diagnosis of training bias depends on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting." pith.science (2026). https://pith.science/paper/6ZQGLIYW

@misc{pith2026241200427,
  author       = {Pith},
  title        = {Pith review of: FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZQGLIYW}},
  note         = {Machine review of arXiv:2412.00427}
}
read the original abstract

In this study, we aim to determine and solve the deficiency of Stable Diffusion Inpainting (SDI) in following the instruction of both prompt and mask. Due to the training bias from masking, the inpainting quality is hindered when the prompt instruction and image condition are not related. Therefore, we conduct a detailed analysis of the internal representations learned by SDI, focusing on how the mask input influences the cross-attention layer. We observe that adapting text key tokens toward the input mask enables the model to selectively paint within the given area. Leveraging these insights, we propose FreeCond, which adjusts only the input mask condition and image condition. By increasing the latent mask value and modifying the frequency of image condition, we align the cross-attention features with the model's training bias to improve generation quality without additional computation, particularly when user inputs are complicated and deviate from the training setup. Extensive experiments demonstrate that FreeCond can enhance any SDI-based model, e.g., yielding up to a 60% and 58% improvement of SDI and SDXLI in the CLIP score.

Figures

Figures reproduced from arXiv: 2412.00427 by the authors.

Figure 1
Figure 1. Comparison of T2I inpainting methods with FreeCond, applied across various mask types: “multi-masks” (column 1 and 4), [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of existing SOTA methods. BrushNet [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Visualization of contextual influence: A random prompt, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: A self-attention visualization in different layers. The [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: A cross-attention visualization of Fig [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: A illustration of the z f c (column 1) and the output lead￾ing by different values of t z (columns 2,3,and 4). The input mask is highlighted by overlaying it onto the output images. To demonstrate that mask input (Ml in this case) signifi￾cantly influences QKT computat…
Figure 10
Figure 10. Figure 10: Illustration of the influence of CFG (w) and each hyperparameter of FreeCond(α, β, γ, tf c), highest values are denoted. Input (15, 1, 0, π, 0)(default) (100, 1, 0, π, 0) (a) (15, 4, 0, π, 0) (b) (15, 1, 0.7, π, 0) (c) (15, 1, 0, 0.75π, 0.5) (d) (15, 2, 0.35, 0.75π, 0…
Figure 11
Figure 11. Figure 11: The qualitative illustration of Fig [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 20 canonical work pages

  1. [1]

    Blended latent diffusion

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM Transactions on Graphics , 42(4):1– 11, 2023. 3

    work page 2023

  2. [2]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 6

    arXiv 2022

  3. [3]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 22560–22570, 2023. 3

    2023

  4. [4]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022. 3

    2022

  5. [5]

    A cat is a cat (not a dog!): Unravel- ing information mix-ups in text-to-image encoders through causal analysis and embedding optimization

    Chieh-Yun Chen, Li-Wu Tsao, Chiang Tseng, and Hong- Han Shuai. A cat is a cat (not a dog!): Unravel- ing information mix-ups in text-to-image encoders through causal analysis and embedding optimization. arXiv preprint arXiv:2410.00321, 2024. 3

    arXiv

  6. [6]

    Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8795–8805, 2024

    work page 2024

  7. [7]

    Z*: Zero-shot style transfer via attention reweighting

    Yingying Deng, Xiangyu He, Fan Tang, and Weiming Dong. Z*: Zero-shot style transfer via attention reweighting. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 6934–6944, 2024

    work page 2024

  8. [8]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    arXiv 2022

Show all 41 references
  1. [9]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3, 8

    2022 arXiv

  2. [10]

    Training-and-prompt-free general painterly harmonization using image-wise attention sharing

    Teng-Fang Hsiao, Bo-Kai Ruan, and Hong-Han Shuai. Training-and-prompt-free general painterly harmonization using image-wise attention sharing. arXiv preprint arXiv:2404.12900, 2024. 3

    2024 arXiv

  3. [11]

    An edit friendly ddpm noise space: Inversion and manipulations

    Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12469– 12478, 2024. 3

    2024

  4. [12]

    In-n-out: Towards good initialization for inpainting and outpainting

    Changho Jo, Woobin Im, and Sung-Eui Yoon. In-n-out: Towards good initialization for inpainting and outpainting. arXiv preprint arXiv:2106.13953, 2021. 3

    2021 arXiv

  5. [13]

    Brushnet: A plug-and-play image inpaint- ing model with decomposed dual-branch diffusion

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpaint- ing model with decomposed dual-branch diffusion. arXiv preprint arXiv:2403.06976, 2024. 2, 3, 4, 7

    2024 arXiv

  6. [14]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3

    2013 arXiv

  7. [15]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 3

    2023

  8. [16]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pages 740–755. Springer, 2014. 2, 4, 7

    2014

  9. [17]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In European Conference on Computer Vision, pages 423–439. Springer, 2022. 3

    2022

  10. [18]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471, 2022. 3

    2022

  11. [19]

    Freeenhance: Tuning- free image enhancement via content-consistent noising-and- denoising process

    Yang Luo, Yiheng Zhang, Zhaofan Qiu, Ting Yao, Zhineng Chen, Yu-Gang Jiang, and Tao Mei. Freeenhance: Tuning- free image enhancement via content-consistent noising-and- denoising process. In Proceedings of the 32nd ACM Interna- tional Conference on Multimedia , pages 7075–7084...

    2024

  12. [20]

    Hd-painter: high-resolution and prompt-faithful text-guided image inpainting with diffusion models

    Hayk Manukyan, Andranik Sargsyan, Barsegh Atanyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Hd-painter: high-resolution and prompt-faithful text-guided image inpainting with diffusion models. arXiv preprint arXiv:2312.14091, 2023. 2, 3, 7

    2023 arXiv

  13. [21]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. arXiv preprint arXiv:2108.01073, 2021. 3

    2021 arXiv

  14. [22]

    Gen- erating diverse structure for image inpainting with hierarchi- cal vq-vae

    Jialun Peng, Dong Liu, Songcen Xu, and Houqiang Li. Gen- erating diverse structure for image inpainting with hierarchi- cal vq-vae. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10775– 10784, 2021. 3 9

    2021

  15. [23]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2, 7

    2023 arXiv

  16. [24]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  17. [25]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning , pages 8821–8831. Pmlr, 2021. 3

    2021

  18. [26]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 2, 3, 7

    2022

  19. [27]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  20. [28]

    Freeu: Free lunch in diffusion u-net

    Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733–4743, 2024. 3, 6

    2024

  21. [29]

    Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting

    Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont- Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J Fleet, Radu Soricut, et al. Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting. In Proceedings of the IEEE/CVF C...

    2023

  22. [30]

    Human preference score: Better aligning text- to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text- to-image models with human preference. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2096–2105, 2023. 3

    2023

  23. [31]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22428–22437, 2023. 3

    2023

  24. [32]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36, 2024. 3

    2024

  25. [33]

    Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model

    Shiyuan Yang, Xiaodong Chen, and Jing Liao. Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3190–3199,

  26. [35]

    Towards understanding the working mechanism of text-to-image dif- fusion model

    Mingyang Yi, Aoxue Li, Yi Xin, and Zhenguo Li. Towards understanding the working mechanism of text-to-image dif- fusion model. arXiv preprint arXiv:2405.15330 , 2024. 3, 6

    2024 arXiv

  27. [36]

    Generative image inpainting with contex- tual attention

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contex- tual attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5505– 5514, 2018. 3

    2018

  28. [37]

    Free-form image inpainting with gated convolution

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4471–4480, 2019

    2019

  29. [38]

    Aggregated contextual transformations for high- resolution image inpainting

    Yanhong Zeng, Jianlong Fu, Hongyang Chao, and Bain- ing Guo. Aggregated contextual transformations for high- resolution image inpainting. IEEE Transactions on Visual- ization and Computer Graphics, 29(7):3266–3280, 2022. 3

    2022

  30. [39]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2, 3, 7

    2023

  31. [40]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018. 3

    2018

  32. [41]

    Pluralistic image completion

    Chuanxia Zheng, Tat-Jen Cham, and Jianfei Cai. Pluralistic image completion. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1438–1447, 2019. 3

    2019

  33. [42]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. arXiv preprint arXiv:2312.03594, 2023. 2, 3, 7 10

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.