Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

DreamLight: Towards Harmonious and Consistent Image Relighting

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DreamLight claims a single diffusion model can relight any subject to match either an image background or a text prompt, without environment maps, while preserving identity and avoiding color bleeding.

desk verdict A solid relighting paper whose headline numbers need a train/test renderer-overlap check before they can be trusted. read the letter →

arxiv 2506.14549 v1 pith:AYDB3PWW submitted 2025-06-17 cs.CV

Yong Liu , Wenpeng Xiao , Qianqian Wang , Junlin Chen , Shiyin Wang , Yitong Wang , Xinglong Wu , Yansong Tang This is my paper
classification cs.CV
keywords imagerelightingdiffusionmodelharmonizationtext-basedlightadapterwavelettransformforegroundconsistencycomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's goal is a single diffusion-based model that relights an extracted foreground so it looks naturally lit inside a new background, with the background supplied either as a photo or as a text prompt. It avoids the expensive path of predicting intrinsics and HDR environment maps, and it avoids the color-bleeding and subject distortion that the authors say plague the current IC-Light baseline. Positioned against image harmonization and inpainting methods, the paper argues that its two modules—a Position-Guided Light Adapter and a Spectral Foreground Fixer—are what allow natural light interaction plus foreground consistency. If correct, this would make photo compositing and virtual production editing cheaper and more flexible: any background description, not just a captured light map, could drive the lighting.

What carries the argument

The Position-Guided Light Adapter (PGLA) is the central mechanism: it encodes the background with a CLIP image encoder, applies a low-frequency spectral filter to suppress texture noise, then uses four sets of learned light-query embeddings, each with an attention-weight mask that decays from one side of the image, to condense background light by direction and inject it only into the foreground via masked cross-attention in the UNet's middle and up blocks. The Spectral Foreground Fixer (SFF) is the supporting mechanism: it uses wavelet transform to split the input subject's high-frequency detail from the relit image's low-frequency color/light, and a small modulator predicts per-pixel coefficients $\alpha,\beta$ to recombine them, preserving identity while adopting the new lighting. Together they carry the paper's claimed trade-off: semantic, position-aware light transfer from the adapter, and distortion-free texture preservation from the fixer.

What would settle it

Render a test set of paired foreground/background images where the only light source is a single directional light angled at 45 degrees (diagonal), and compare DreamLight's PSNR and LPIPS against the same model evaluated on axis-aligned lights of equal intensity. If diagonal relighting fails badly or attention analysis shows the four directional queries converge to identical responses, then the four-axis assumption is load-bearing and the universal claim collapses.

Watch

Extended reading notes

Core claim

DreamLight claims that universal relighting of natural images can be done by a single diffusion model without environment maps, by reorganizing inputs into a unified foreground-background-text format and letting a pretrained diffusion prior generate the lit composite. The model treats background lighting as four directional components (left, right, top, down), condenses them into learnable light-query embeddings through direction-biased masked attention, and modulates the foreground so different object regions respond to nearby light sources. A separate Spectral Foreground Fixer then recombines the high-frequency textures of the original subject with the low-frequency light of the relit result using wavelet decomposition and learned per-pixel coefficients. The paper reports that this design outperforms image-based relighting baselines on PSNR/SSIM/LPIPS/CLIP-IS (e.g., PSNR 22.15 vs 20.27 for IC-Light) and text-based relighting baselines on CLIP/aesthetic/Image Reward scores, with the two-condition guidance emerging as an un-trained bonus.

Load-bearing premise

The load-bearing premise, stated in Section 3.2, is that all scene lighting can be reduced to four axis-aligned directions (left, right, top, down); if lighting is diagonal, ambient, or dominated by multiple local sources, the adapter has no representational room to capture it.

Editorial extensions

If this is right

  • A single checkpoint can serve both image-conditioned and text-conditioned relighting, and can even combine both conditions at inference without extra training.
  • Relighting no longer requires HDR environment maps or intrinsic decomposition training data; ordinary images and prompts suffice.
  • Because PGLA only modifies middle and up UNet blocks, the global semantics of the composite stay stable while lighting adapts, reducing distortion risk.
  • SFF-type wavelet re-composition can be applied after other diffusion editing methods to preserve small high-frequency regions like faces and text.
  • On the paper's 600-pair Arnold-rendered benchmark, the reported gains suggest near-photorealistic lighting transfer may be attainable from coarse directional cues alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The four-direction assumption suggests the model should be tested on diagonal, ambient, or multi-source lighting; a natural extension is to increase the number or orientation of light queries beyond four, or make them continuous angles.
  • The authors report data generated partly by IC-Light and by a bootstrapped relighting LoRA; if those synthetic pairs inherit systematic biases (e.g., IC-Light's color casts), DreamLight's advantage may partly reflect the training distribution rather than the architecture.
  • The emergent dual-condition behavior hints that the same adapter could support fine-grained lighting control, e.g., user-provided light direction vectors or scribbles, by treating them as additional queries.
  • The SFF's assumed split of high-frequency as texture and low-frequency as light is approximate; on glossy or specular objects, specular highlights live in high frequencies and may be wrongly preserved as identity texture rather than re-lit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. DreamLight proposes a single diffusion-based model for universal image relighting: compositing a subject into either a provided background image (image-based relighting) or a text-described background (text-based relighting), without requiring environment maps. The subject is extracted with a segmentation model, and the masked foreground latents, background latents, and random noise are concatenated as UNet input. Two modules are introduced: the Position-Guided Light Adapter (PGLA), which encodes the background with a CLIP image encoder, applies a low-frequency spectral filter, learns four sets of light queries for the left/right/top/down directions, and modulates foreground features via direction-biased masked cross-attention in the middle and up blocks of the UNet; and the Spectral Foreground Fixer (SFF), a separately trained wavelet-domain post-processor that recombines the high-frequency textures of the input subject with the low-frequency lighting of the relighted result. Training data of roughly 1.05M pairs comes from three pipelines: a bootstrapped relighting-LoRA process (600k), Arnold Renderer renders of 3D assets (150k), and IC-Light-processed pairs (300k). On a 600-pair Arnold-rendered benchmark, the paper reports gains over INR, PCT, IH, and IC-Light in PSNR/SSIM/LPIPS/CLIP-IS (Table 1) and over PowerPaint, BrushNet, and IC-Light in CLIP score, aesthetic score, and Image Reward (Table 2), with ablations supporting the adapter and fixer designs.

Significance. If the reported numbers are taken at face value, this is a solid and useful contribution to image relighting and harmonization. The unified treatment of image-based and text-based relighting in a single model is a genuine step beyond IC-Light's two-model design; the PGLA is an original mechanism for injecting directional lighting information without intrinsic decomposition or HDR maps; and the SFF is a simple, well-ablated post-processing idea that visibly improves subject identity preservation, with Table 4 showing a large gain on cropped face regions. The paper also ships substantial engineering: over one million training pairs from three pipelines, a promised open-source relighting LoRA, and an evaluation protocol with standard metrics, all of which are reproducible commitments. However, the evaluation chain has two weaknesses that bear directly on the central claim: the 600-pair benchmark is rendered with the same renderer used to build 150k of the training pairs with no stated disjointness, and the primary baseline (IC-Light) also appears as a generator of 300k training pairs, so the headline margin in Table 1 is at least partly confounded.

major comments (3)
  1. [4.1 with 3.4] The central quantitative claim (Table 1, PSNR 22.15 vs 20.27 for IC-Light) is measured on a 600-pair benchmark that is 'rendered by Arnold Renderer from real objects' (Section 4.1), while the training set contains 150k pairs generated by 'an automatically rendering pipeline on 3D Arnold Renderer' with random light sources and HDR images (Section 3.4). The manuscript never states whether the benchmark objects, HDR maps, light configurations, and renderer settings are disjoint from the training pipeline, so the reported margin may partly reflect familiarity with Arnold's light transport and texture statistics rather than general relighting quality. Please state the disjointness control explicitly, and add either a cross-renderer evaluation (e.g., held-out assets rendered in Blender or Octane) or a qualitative real-photograph comparison to support the claim of 'universal relighting of natural images' in Section 4.2.
  2. [3.4 with Table 1] Section 3.4 states that 300k training pairs were produced by 'process[ing] vanilla images with IC-Light [22]', and Table 1 then compares DreamLight against IC-Light. Because the model has been trained on IC-Light's outputs, the comparison is confounded: the reported margin over IC-Light measures improvement relative to a baseline that also appears inside the training data, so the contribution of the proposed components is not cleanly isolated. Please report the image-based metrics separately for test items generated with each of the three data-generation pipelines, or on a clean external benchmark, and state explicitly whether any IC-Light-derived pair overlaps with the 600-pair evaluation set.
  3. [3.2 with 4.3] Section 3.2 assumes that 'the light sources can be split into four basic directions: left, right, top, and down', which gives the PGLA no explicit representational room for diagonal, ambient, or multiple local light sources and sits uneasily with the claim of universal relighting. The impact is mitigated because the background latents are also concatenated into the UNet input, so the direction queries are an auxiliary conditioning channel rather than the only source of lighting information; nevertheless, the attribution of the Table 3 gains to direction-biased attention would be more convincing with an ablation that varies the number of directions (e.g., 8) or an evaluation restricted to test pairs whose dominant light direction is diagonal.
minor comments (6)
  1. [Table 2 caption] The caption 'PP denotes PowerPoint' should read 'PowerPaint'; the rows PP-V1 and PP-V2 refer to the PowerPaint variants of reference [39].
  2. [3.4] There are two typos in Section 3.4: 'relighting ominicontrol [44] lora' should be 'Ominicontrol', and 'LLaV A [48]' should be 'LLaVA'.
  3. [Equation (1)] The low-frequency filter is specified only as a Gaussian with cutoff frequency σ; please give the functional form (e.g., standard deviation versus cutoff) and state the shape of the decay map in Figure 3 (linear, cosine, or learned), since these are free parameters of the adapter.
  4. [4.2] The abstract claims that a user study demonstrates the performance, but the main text only says that 'Results of user study are reported in the supplementary materials'; the paper is not self-contained on this point, so please include the user-study protocol, stimuli, and statistics in the main text or explicitly state that the claim rests on the supplementary report.
  5. [Equation (2)] Please clarify how the wavelet subbands are aggregated in Equation (2), since α and β are defined as H×W×3 maps while the wavelet transform produces multiple subbands per color channel.
  6. [3.1] The reference to 'The green and brown lines' in Figure 2 is not reproducible in grayscale printing; please use distinct line styles or labels in addition to color.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; empirical claims rest on an external benchmark, with renderer-overlap and baseline-contamination as non-circular validity risks.

full rationale

The paper's central claim is empirical: DreamLight outperforms IC-Light and other relighters on a 600-pair benchmark (Table 1: PSNR 22.15 vs 20.27). No step in the derivation chain reduces to its own input by construction. PGLA is an architectural adapter with learnable queries and hand-designed direction masks; it is an ansatz supported by ablations (Table 3), not a definition of the evaluation metric. SFF is trained self-supervised on pseudo color-transform pairs and then applied as post-processing; again, its training objective (MSE + perceptual loss) is not the same as the Table 4 face-crop metrics, so the reported improvement is not a fitted parameter renamed as a prediction. The four-direction light decomposition is an explicit simplifying assumption, not a disguised restatement of the benchmark. The training-data pipeline does include IC-Light-generated pairs (Section 3.4) and the test benchmark uses the same Arnold Renderer as some training data (Sections 3.4 and 4.1); these are legitimate threats to the external-validity and fairness of the numerical comparisons, but they are not circularity: no equation, fitted value, or self-citation forces the reported PSNR/LPIPS scores. The paper contains no load-bearing self-citations, no imported uniqueness theorems, and no renaming of a known result as a new derivation. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard deep learning components (diffusion, CLIP, VAE) plus a small number of hand-set hyperparameters (cutoff frequency, query count) and a strong modeling assumption about four lighting directions. No new physical entities are introduced. The main novelty is in the architectural wiring, not in new parameters with external predictive power.

free parameters (3)
  • Cutoff frequency sigma = 5
    Hand-set in the low-frequency enhancement filter (Eq. 1) to emphasize lighting and color tone while suppressing high-frequency background texture.
  • Number of light queries per direction = 4
    Architectural choice in PGLA; each direction uses 4 learnable query embeddings.
  • Decay map shape = linear decay (not specified)
    The direction-biased attention relies on hand-designed coefficient maps that decay from one side of the background feature to the other; the exact functional form is not reported.
assumptions (5)
  • domain assumption Light sources can be split into four basic directions: left, right, top, and down.
    Introduced in Section 3.2 to define the direction-biased queries; restricts universality.
  • standard math FFT and wavelet transforms correctly separate high- and low-frequency image components.
    Used in PGLA (Eq. 1) and SFF (Eq. 2); these are standard signal processing results.
  • domain assumption Pretrained VAE and CLIP encoders retain sufficient lighting and identity information for relighting.
    The pipeline uses StableDiffusion-v1.5 and CLIP-H without fine-tuning; the method depends on these latent spaces.
  • domain assumption RMBG-1.4 segmentation mask is accurate for all inputs.
    The foreground mask drives both the diffusion input and the SFF composition; segmentation errors directly corrupt the output.
  • domain assumption Bootstrapped relighting LoRA and IC-Light-generated training pairs are unbiased and of sufficient quality.
    Part of the training data is produced by models that also serve as baselines, which may introduce artifacts or favor certain lighting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DreamLight: Towards Harmonious and Consistent Image Relighting." pith.science (2026). https://pith.science/paper/AYDB3PWW

@misc{pith2026250614549,
  author       = {Pith},
  title        = {Pith review of: DreamLight: Towards Harmonious and Consistent Image Relighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYDB3PWW}},
  note         = {Machine review of arXiv:2506.14549}
}
read the original abstract

We introduce a model named DreamLight for universal image relighting in this work, which can seamlessly composite subjects into a new background while maintaining aesthetic uniformity in terms of lighting and color tone. The background can be specified by natural images (image-based relighting) or generated from unlimited text prompts (text-based relighting). Existing studies primarily focus on image-based relighting, while with scant exploration into text-based scenarios. Some works employ intricate disentanglement pipeline designs relying on environment maps to provide relevant information, which grapples with the expensive data cost required for intrinsic decomposition and light source. Other methods take this task as an image translation problem and perform pixel-level transformation with autoencoder architecture. While these methods have achieved decent harmonization effects, they struggle to generate realistic and natural light interaction effects between the foreground and background. To alleviate these challenges, we reorganize the input data into a unified format and leverage the semantic prior provided by the pretrained diffusion model to facilitate the generation of natural results. Moreover, we propose a Position-Guided Light Adapter (PGLA) that condenses light information from different directions in the background into designed light query embeddings, and modulates the foreground with direction-biased masked attention. In addition, we present a post-processing module named Spectral Foreground Fixer (SFF) to adaptively reorganize different frequency components of subject and relighted background, which helps enhance the consistency of foreground appearance. Extensive comparisons and user study demonstrate that our DreamLight achieves remarkable relighting performance.

Figures

Figures reproduced from arXiv: 2506.14549 by the authors.

Figure 1
Figure 1. Our DreamLight is a unified image relighting model capable of performing relighting [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of our DreamLight. It takes the foreground image, background image, and text [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Process of direction-biased masked attention. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Process of Spectral Foreground Fixer. The modulator is utilized to predict a set of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Image-based relighting results. (a) and (b) demonstrate the comparisons with popular [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Results of text-based relighting. PowerPaint [ [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Visualization comparisons with different light adapter design strategies. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization comparisons with different foreground fixer strategies. For the ease of [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Visualization results of our DreamLight condi [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A text-prompted perceptual metric (TPIPS) trained on a new human-judgment dataset matches human aspect-conditioned similarity choices better than existing VLMs and prior metrics.

  2. Consistent Feature Transport for Image Relighting

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A training objective for rectified-flow image editing that supervises lighting transport with cross-instance pairs and a synthetic portrait relighting dataset improves relighting metrics in the paper's experiments.

Reference graph

Works this paper leans on

53 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [22]

    Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport. InICLR, 2025

    work page 2025

  2. [1]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InCVPR, pages 10684–10695, 2022

    2022

  3. [2]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  4. [3]

    Denoising diffusion probabilistic models.NeurIPS, pages 6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.NeurIPS, pages 6840–6851, 2020

    work page 2020

  5. [4]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    arXiv 2010

  6. [5]

    Anydoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. InCVPR, pages 6593–6602, 2024

    work page 2024

  7. [6]

    Com- poser: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Com- poser: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    arXiv 2023

  8. [7]

    Paint by example: Exemplar-based image editing with diffusion models

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion models. In CVPR, pages 18381–18391, 2023

    2023

Show all 53 references
  1. [8]

    Relightful harmonization: Lighting-aware portrait background replacement

    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. InCVPR, pages 6452–6462, 2024

    2024

  2. [9]

    Stylitgan: Image-based relighting via latent control

    Anand Bhattad, James Soole, and DA Forsyth. Stylitgan: Image-based relighting via latent control. InCVPR, pages 4231–4240, 2024

    2024

  3. [10]

    Switch- light: Co-design of physics-driven architecture and pre-training framework for human portrait relighting

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switch- light: Co-design of physics-driven architecture and pre-training framework for human portrait relighting. InCVPR, pages 25096–25106, 2024

    2024

  4. [11]

    Lightpainter: interactive portrait relighting with freehand scribble

    Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M Patel. Lightpainter: interactive portrait relighting with freehand scribble. InCVPR, pages 195–205, 2023

    2023

  5. [12]

    Holo-relighting: Controllable volumetric portrait relighting from a single image

    Yiqun Mei, Yu Zeng, He Zhang, Zhixin Shu, Xuaner Zhang, Sai Bi, Jianming Zhang, HyunJoon Jung, and Vishal M Patel. Holo-relighting: Controllable volumetric portrait relighting from a single image. InCVPR, pages 4263–4273, 2024

    2024

  6. [13]

    Total relighting: learning to relight portraits for background replacement.ACM Trans

    Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E Debevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement.ACM Trans. Graph., pages 43–1, 2021

    2021

  7. [14]

    Intrinsic image harmonization

    Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, and Bing Zheng. Intrinsic image harmonization. InProceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 16367–16376, 2021

    2021

  8. [15]

    Intrinsic harmonization for illumination-aware image compositing

    Chris Careaga, S Mahdi H Miangoleh, and Ya ˘gız Aksoy. Intrinsic harmonization for illumination-aware image compositing. InSIGGRAPH Asia, pages 1–10, 2023

    2023

  9. [16]

    Pct-net: Full resolution image harmonization using pixel-wise color transformations

    Julian Jorge Andrade Guerreiro, Mitsuru Nakazawa, and Björn Stenger. Pct-net: Full resolution image harmonization using pixel-wise color transformations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5917–5926, 2023

    2023

  10. [17]

    Dense pixel-to-pixel harmonization via continuous image representation.IEEE Transactions on Circuits and Systems for Video Technology, 2023

    Jianqi Chen, Yilan Zhang, Zhengxia Zou, Keyan Chen, and Zhenwei Shi. Dense pixel-to-pixel harmonization via continuous image representation.IEEE Transactions on Circuits and Systems for Video Technology, 2023. 10

    2023

  11. [18]

    Dccf: Deep comprehensible color filter learning framework for high-resolution image harmonization

    Ben Xue, Shenghui Ran, Quan Chen, Rongfei Jia, Binqiang Zhao, and Xing Tang. Dccf: Deep comprehensible color filter learning framework for high-resolution image harmonization. In ECCV, pages 300–316, 2022

    2022

  12. [19]

    High-resolution image harmonization via collaborative dual transformations

    Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, and Liqing Zhang. High-resolution image harmonization via collaborative dual transformations. InCVPR, pages 18470–18479, 2022

    2022

  13. [20]

    Neural gaffer: Relighting any object via diffusion.arXiv preprint arXiv:2406.07520, 2024

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion.arXiv preprint arXiv:2406.07520, 2024

    2024 arXiv

  14. [21]

    Dilightnet: Fine-grained lighting control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained lighting control for diffusion-based image generation. InSIGGRAPH, pages 1–12, 2024

    2024

  15. [23]

    Learning physics-guided face relighting under directional light

    Thomas Nestmeyer, Jean-François Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics-guided face relighting under directional light. InCVPR, pages 5124–5133, 2020

    2020

  16. [24]

    Difareli: Diffu- sion face relighting

    Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supasorn Suwajanakorn. Difareli: Diffu- sion face relighting. InICCV, pages 22646–22657, 2023

    2023

  17. [25]

    Single image portrait relighting.ACM Transactions on Graphics (TOG), 38(4):1–12, 2019

    Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi. Single image portrait relighting.ACM Transactions on Graphics (TOG), 38(4):1–12, 2019

    2019

  18. [26]

    Neural video portrait relighting in real-time via consistency modeling

    Longwen Zhang, Qixuan Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Neural video portrait relighting in real-time via consistency modeling. InICCV, pages 802–812, 2021

    2021

  19. [27]

    Composite photograph harmo- nization with complete background cues

    Yazhou Xing, Yu Li, Xintao Wang, Ye Zhu, and Qifeng Chen. Composite photograph harmo- nization with complete background cues. InACM MM, pages 2296–2304, 2022

    2022

  20. [28]

    Acquiring the reflectance field of a human face

    Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. InSIGGRAPH, pages 145–156, 2000

    2000

  21. [29]

    Cut-and-paste object insertion by enabling deep image prior for reshading

    Anand Bhattad and David A Forsyth. Cut-and-paste object insertion by enabling deep image prior for reshading. In3DV, pages 332–341, 2022

    2022

  22. [30]

    Semi-supervised parametric real-world image harmonization

    Ke Wang, Michaël Gharbi, He Zhang, Zhihao Xia, and Eli Shechtman. Semi-supervised parametric real-world image harmonization. InCVPR, pages 5927–5936, 2023

    2023

  23. [31]

    Image harmonization with transformer

    Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, and Junyu Dong. Image harmonization with transformer. InICCV, pages 14870–14879, 2021

    2021

  24. [32]

    Ssh: A self-supervised framework for image harmonization

    Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, and Zhangyang Wang. Ssh: A self-supervised framework for image harmonization. InICCV, pages 4832–4841, 2021

    2021

  25. [33]

    Deep image harmonization in dual color spaces

    Linfeng Tan, Jiangtong Li, Li Niu, and Liqing Zhang. Deep image harmonization in dual color spaces. InProceedings of the 31st ACM International Conference on Multimedia, pages 2159–2167, 2023

    2023

  26. [34]

    Harmonizer: Learning to perform white-box image and video harmonization

    Zhanghan Ke, Chunyi Sun, Lei Zhu, Ke Xu, and Rynson WH Lau. Harmonizer: Learning to perform white-box image and video harmonization. InECCV, pages 690–706, 2022

    2022

  27. [35]

    Dovenet: Deep image harmonization via domain verification

    Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang. Dovenet: Deep image harmonization via domain verification. InCVPR, pages 8394–8403, 2020

    2020

  28. [36]

    Deep image harmonization

    Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, and Ming-Hsuan Yang. Deep image harmonization. InCVPR, pages 3789–3797, 2017. 11

    2017

  29. [37]

    Lasagna: Layered score distillation for disentangled object relighting

    Dina Bashkirova, Arijit Ray, Rupayan Mallick, Sarah Adel Bargal, Jianming Zhang, Ranjay Krishna, and Kate Saenko. Lasagna: Layered score distillation for disentangled object relighting. arXiv preprint arXiv:2312.00833, 2023

    2023 arXiv

  30. [38]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion.arXiv preprint arXiv:2403.06976, 2024

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion.arXiv preprint arXiv:2403.06976, 2024

    2024 arXiv

  31. [39]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting.arXiv preprint arXiv:2312.03594, 2023

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting.arXiv preprint arXiv:2312.03594, 2023

    2023 arXiv

  32. [40]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. InCVPR, pages 18392–18402, 2023

    2023

  33. [41]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  34. [42]

    Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. InICCV, pages 22560–22570, 2023

    2023

  35. [43]

    Perceptual losses for real-time style transfer and super-resolution

    Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. InECCV, pages 694–711, 2016

    2016

  36. [44]

    Ominicontrol: Minimal and universal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024

    2024 arXiv

  37. [45]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InCVPR, pages 13142–13153, 2023

    2023

  38. [46]

    Laion- 5b: An open large-scale dataset for training next generation image-text models.NeurIPS, pages 25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models.NeurIPS, pages 25278–25294, 2022

    2022

  39. [47]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  40. [48]

    Visual instruction tuning.NeurIPS, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS, 2024

    2024

  41. [49]

    Pytorch: An imperative style, high-performance deep learning library.NeurIPS, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.NeurIPS, 2019

    2019

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InICML, pages 8748–8763, 2021

    2021

  43. [51]

    The un- reasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The un- reasonable effectiveness of deep features as a perceptual metric. InCVPR, pages 586–595, 2018

    2018

  44. [52]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. NeurIPS, 2024

    2024

  45. [53]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InICCV, pages 3836–3847, 2023. 12

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.