Pith. sign in

REVIEW 4 major objections 4 minor 32 references

Multi-view Image Diffusion via Coordinate Noise and Fourier Attention

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that multi-view diffusion images become more consistent when noise is initialized with shared coordinate structure and attention is filtered by spatial frequency, especially in non-overlapping regions.

desk verdict Solid combination of known ideas for multi-view consistency, but the paper's own ablations don't isolate the Fourier attention block as the source of its gains. read the letter →

arxiv 2412.03756 v1 pith:WZSLM6H3 submitted 2024-12-04 cs.CV cs.AI

Justin Theiss , Norman Müller , Daeil Kim , Aayush Prakash This is my paper · ORCID
classification cs.CVcs.AI
keywords multi-viewconsistencytext-to-imagediffusioncoordinatenoiseinitializationFourier-basedattentionpanoramicimagegenerationdepth-to-imagecross-attentionlosslatentmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models can now draw a single image from text, but drawing several views of the same scene usually fails: things in one view do not match things in another, especially in areas where the views do not overlap. This paper claims that the failure can be fixed by changing how generation starts and what the attention layers look at. It initializes each view's noise with shared random noise blended with low-frequency coordinate and depth structure, adds an attention block that in non-overlapping regions attends to progressively higher spatial frequencies as denoising proceeds, and adds a loss that pins prompt-to-image attention to ground-truth layout. If the claim holds, panoramic and depth-conditioned generation becomes visibly more coherent, with textures and objects staying put across large view changes; the paper reports overlap-PSNR ratio rising from 0.60 to 0.66 in panoramas and from 0.87 to 0.94 in depth-to-image. The recipe works on top of an existing latent diffusion model without retraining its backbone, so the fix is portable.

What carries the argument

The load-bearing object is the coordinate-noise initialization of Eq. 8--9: a per-view linear blend $w c^i + (1-w)\epsilon_{\mathrm{shared}}$ of shared Gaussian noise with low-frequency scene information (normalized depth maps, or pixel coordinates warped into the center view). The Fourier-based attention (FBA) block is the mechanism that exploits it: it takes U-Net features produced from coordinate noise, applies a 2D FFT, multiplies by the high-pass mask $M_{r_t}^F$ with radius $r_t = 1 - t/T$, applies the inverse FFT plus a positional encoding, and merges these filtered non-overlap features with correspondence-aware features from the overlapping-region mask before softmax attention. The prompt cross-attention loss $L_{\mathrm{XA}}^l = \|M_t^l - M_0^l\|$ anchors prompt-to-view attention to clean-latent attention maps. The FFT mask does the central work: it turns the low-frequency bias of the initialization into a time-dependent curriculum from broad structure to fine detail.

What would settle it

Run the panorama experiment with the Fourier attention mask replaced by a fixed random mask with the same size and total energy while keeping coordinate noise and the cross-attention loss; if the overlap-PSNR ratio and intra-LPIPS stay near 0.66 and 0.75, the frequency-selection mechanism is not carrying the claimed alignment. Alternatively, measure whether the attention weights in non-overlapping regions correlate with ground-truth scene layout when ground truth is available; if they do not, the Fourier-based attention blocks are not actually aligning structure.

Watch

Extended reading notes

Core claim

The paper's central claim is that multi-view consistency improves when noise initialization and attention are both made aware of the scene's spatial structure. Instead of sampling independent Gaussian noise per view, the method uses coordinate noise: per-view latents built as $\hat{z}_T^i = \sqrt{\bar{\alpha}_T}\hat{\epsilon}^i + \sqrt{1-\bar{\alpha}_T}\epsilon^i$ with $\hat{\epsilon}^i = w c^i + (1-w)\epsilon_{\mathrm{shared}}$, where $c^i$ is normalized depth or pixel coordinates transformed into a reference view. The Fourier-based attention block then gathers U-Net features generated from this coordinate noise, applies an FFT, keeps only frequencies above a time-dependent radius ($r_t = 1-t/T$), and combines those filtered features with correspondence-aware features from overlapping regions before softmax attention. A prompt cross-attention loss, $L_{\mathrm{XA}}^l = \|M_t^l - M_0^l\|$, anchors each view's prompt-to-image attention maps to the noise-free ground-truth maps. On the paper's measurements this yields overlap-PSNR ratios of 0.66 in panoramas and 0.94 in depth-to-image, up from 0.60 and 0.87 for the MVDiffusion baseline, and intra-LPIPS decreases from 0.80 to 0.75 in panoramas.

Load-bearing premise

Everything rests on the premise that blending shared random noise with low-frequency coordinate and depth maps actually plants consistent scene layout into the model's internal representations, so that the frequency-filtered attention aligns real structure rather than a shared random signal.

Editorial extensions

If this is right

  • Panorama generation becomes more globally coherent: the overlapping-PSNR ratio rises from 0.60 to 0.66 and intra-LPIPS falls from 0.80 to 0.75, meaning colors and textures match better across the full sweep.
  • Depth-conditioned multi-view generation improves from a 0.87 to a 0.94 overlap-PSNR ratio, so small objects, large objects, and room-scale environment textures change less from view to view.
  • Prompt fidelity improves: scenes generated from prompts like "a house with a pool in the backyard" actually contain both the house and the pool, because the cross-attention loss pins prompt-to-image attention to ground-truth layout.
  • The three components are additive: shared noise alone helps, coordinate noise helps more, adding the Fourier-based attention blocks helps further, and the full model with the cross-attention loss gives the best numbers in the paper's ablation.
  • Because the U-Net backbone is frozen and only new attention blocks are trained, the recipe can be layered onto an existing single-view latent diffusion model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the time-dependent frequency schedule suggests a testable extension in which the radius $r_t$ is matched to the noise level $\bar{\alpha}_t$ rather than a linear schedule; the paper does not test this variant.
  • Inference: coordinate noise requires only camera pose and optional depth, so the same initialization could be dropped into video-diffusion and 3D generation pipelines to stabilize global appearance without retraining the backbone.
  • Inference: the cross-attention loss needs ground-truth multi-view images to anchor the attention maps, which limits it to datasets with posed multi-view captures; in unposed settings, a self-supervised anchor such as prediction agreement across views would be a natural replacement.
  • Inference: the reported depth-to-image FID is slightly worse than the main baseline, which the authors attribute to imprecise per-view prompts; a testable implication is that with better cross-view-consistent captions, the FID gap would close or reverse.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. arXiv:2412.03756 addresses multi-view text-to-image consistency in latent diffusion models with three proposed components. A coordinate-noise initialization (Eqs. 7-9) replaces pure Gaussian latents with a per-view blend of normalized coordinates or depth maps, shared Gaussian noise, and independent noise. Fourier-based attention blocks (FBA; Eqs. 10-16) attend in overlapping regions via MVDiffusion-style correspondence-aware attention and in non-overlapping regions to features filtered by a time-dependent frequency mask. A prompt cross-attention loss (Eq. 17) matches noisy-latent cross-attention maps to clean-latent maps. Experiments on Matterport3D panoramic generation and ScanNet depth-to-image generation report improved overlap PSNR ratio and intra-LPIPS relative to MVDiffusion, with mixed FID/CLIP results (better in panoramas, slightly worse in depth-to-image). The paper includes ablations of the noise weight and frequency-mask schedule in the supplement.

Significance. If the reported gains are reproducible, the paper is a solid engineering contribution: the method is modular, trains only added blocks while freezing a pretrained U-Net, and the supplement contains careful ablations of several design choices. The quantitative improvements over MVDiffusion on overlap PSNR ratio and intra-LPIPS are plausible and appear consistent across two datasets. The main weaknesses are that the headlined non-overlapping-region mechanism is not evaluated directly and that no uncertainty estimates are provided; both are required to support the SOTA claim.

major comments (4)
  1. [Sec. 3.3 and Table 4] The central claim that FBA improves multi-view consistency 'particularly in non-overlapping regions' (Sec. 1, Sec. 3.3) is not supported by the quantitative evidence. The reported Overlap PSNR ratio is, by construction, computed only in overlapping regions, and Intra-LPIPS is an average over all generated view pairs without spatial localization. In the ablation of Table 4, adding FBA Blocks (row 3: PSNR 23.9, ratio 0.64, I-LPIPS 0.78) gives essentially the same consistency numbers as Coordinate Noise alone (row 2: 24.2, 0.65, 0.78); the largest improvement appears only after the cross-attention loss is added in the Full Model (row 4: 24.7, 0.66, 0.75). Thus the specific causal role of FBA in non-overlapping regions rests on the qualitative Figures 4-5, and the measured gains could plausibly arise from the shared-noise initialization alone. Please add a non-overlap-specific metric (e.g., PSNR or LPIPS restricted to masked non-overlap regions) and an ablation that separates FBA from the cross-attention loss.
  2. [Sec. 4.3, Tables 1-4] All comparisons are reported as single point estimates with no error bars, number of independent runs, or statistical tests. Several headline differences are modest (e.g., overlap PSNR ratio 0.66 vs. 0.60 in Table 2; I-LPIPS 0.75 vs. 0.79; Table 3 ratio 0.94 vs. 0.87), and the ablation margins in Table 4 are even smaller. Because multi-view generation is stochastic and only 8 views are generated per scene, these margins may be within run-to-run variation. Please report at least three independent sampling runs with mean and standard deviation (or confidence intervals) for the main metrics, and ideally for the ablations.
  3. [Appendix C, Tables S2 and S4] The values w=0.5 and the HPF-rt schedule used for the main results are selected by ablations on the same Matterport3D evaluation protocol used for the headline numbers (Tables S2 and S4). This is a form of test-set selection and can inflate the reported improvements over baselines. Please either reserve a validation split for hyperparameter selection, or explicitly report the selection protocol and quantify the sensitivity of the main comparisons to w and rt.
  4. [Eqs. 12-14] The time-dependent mask defined by rt = 1 - t/T and Mrt_F = 1 - 1_{central} has a degenerate behavior at the final denoising steps. At t=T (noisiest), rt=0 and M is all ones, so all frequencies pass; at t=1 (cleanest), rt≈1 and M is all zeros, so the frequency-filtered feature term F^{-1}(M⊙F(G)) in Eq. 14 vanishes, leaving only the positional-encoding term γ(1−rt). This is in tension with the stated design of attending to progressively higher spatial frequencies across denoising time steps and makes the attention target at the final step essentially position-only. Please clarify whether FBA is intentionally disabled near t=1, and if not, define rt so that the pass band does not collapse (e.g., rt=(t-1)/(T-1)).
minor comments (4)
  1. [Sec. 3.3, Eqs. 11 and 14] The positional encoding γ(·) is used for both the displacement in Eq. 11 and the radius in Eq. 14, but it is never defined. Please specify the encoding (e.g., sinusoidal positional encoding with which frequencies) and note whether the two uses share weights.
  2. [Sec. 3.4, Eq. 17] Please clarify what Ml_t and Ml_0 represent exactly: normalized cross-attention distributions per prompt token, and which layers (and heads, if any) are aggregated. Also state the norm used in Eq. 17.
  3. [Sec. 4.3.2 and Sec. 5] The conclusion states that the method 'outperform[s] SOTA baselines' and 'achieve[s] multi-view consistency while maintaining the diversity,' but Table 3 shows a worse FID than MVDiffusion (27.0 vs. 23.7). The main text qualifies this in Sec. 4.3.2, but the conclusion should be revised to avoid overstatement.
  4. [Sec. 4.2] For SyncDiffusion, the table shows '-' for overlap PSNR; please explain explicitly that the comparison is computed after splitting the single generated panorama into non-overlapping views, making overlap PSNR undefined.

Circularity Check

1 steps flagged · score 2.0 of 10

No equation-level circularity; mild fitted-hyperparameter reporting on the same evaluation benchmarks.

  1. fitted input called prediction [Supplementary Material, Tables S2 and S4 (ablation of coordinate-noise weight w and frequency-mask schedule rt)]
    "The results shown in Table S2 indicate that setting the weight w = 0.5 indeed provides the optimal result. ... our method of selecting the full spectrum of spatial frequencies for attention at noisier time steps and high spatial frequencies at less noisy time steps (i.e. "HPF-rt") provides the best overall performance, particularly for FID and overlapping PSNR."

    The coordinate-noise weight w (Eq. 8) and the time-dependent mask schedule rt (Eqs. 12-13) are selected by optimizing the headline metrics (FID, PSNR, Ratio, Intra-LPIPS) on the evaluation benchmarks, and the final model then reports "improves SOTA" on the same benchmarks (Tables 1-4). The reported advantage is therefore partly a selection artifact on the test set rather than an independently predicted result. This is a mild fitted-input issue rather than a derivation that reduces by construction; the model equations themselves are not derived from the metrics.

full rationale

I walked the derivation chain: Eq. 8/9 define coordinate noise as a linear blend of coordinates/depth and shared Gaussian noise; Eq. 14 applies a time-dependent frequency mask to features obtained from that noise; Eq. 17 enforces agreement with clean-latent cross-attention maps. None of these is equivalent by construction to the reported consistency metrics, and there are no load-bearing self-citations or imported uniqueness theorems. The FBA ablation (Table 4) honestly shows FBA alone is similar to coordinate noise, with the main gain from the XA loss, so the central attribution is underdetermined but not circular. The only circularity-adjacent concern is that w and the mask schedule were tuned on the same evaluation set used for the final quantitative claims (Tables S2/S4 vs. Tables 1-4), which is a mild fitted-input reporting issue rather than an equation-level tautology. Hence score 2.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard diffusion math plus three domain assumptions about the noise-initialization gap, the usefulness of ground-truth attention maps, and the reliability of homography-based overlap masks. The main free parameters are hyperparameters selected by ablation on the same benchmarks used for evaluation, which weakens the strength of the reported gains.

free parameters (3)
  • coordinate noise weight w = 0.5
    Chosen by ablation on the panoramic benchmark (Table S2) to optimize FID and PSNR; used in Eq. 8.
  • cross-attention loss weight lambda = 10
    Set in Eq. 18 without ablation; affects the balance between denoising and attention alignment.
  • frequency mask radius schedule rt = rt = 1 - t/T
    Defined in Eq. 12 as a linear schedule and validated as best among four alternatives in Table S4.
assumptions (4)
  • standard math Latent diffusion forward process with fixed variance schedule (Eqs. 1-3)
    Assumed as background for the diffusion model.
  • domain assumption Training/inference SNR gap described by Lin et al. [13] and Zhang et al. [29]
    Motivates injecting low-frequency information at initialization; the gap is a known empirical observation, not derived here.
  • domain assumption Ground-truth cross-attention maps Ml_0 from clean latents are a valid training target
    Used for the loss in Eq. 17; the paper assumes these maps encode spatial relationships worth matching.
  • domain assumption Overlap masks Mi,j_ovr from homographies are accurate
    Used to separate overlapping and non-overlapping regions in Eq. 15.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-view Image Diffusion via Coordinate Noise and Fourier Attention." pith.science (2026). https://pith.science/paper/WZSLM6H3

@misc{pith2026241203756,
  author       = {Pith},
  title        = {Pith review of: Multi-view Image Diffusion via Coordinate Noise and Fourier Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZSLM6H3}},
  note         = {Machine review of arXiv:2412.03756}
}
read the original abstract

Recently, text-to-image generation with diffusion models has made significant advancements in both higher fidelity and generalization capabilities compared to previous baselines. However, generating holistic multi-view consistent images from prompts still remains an important and challenging task. To address this challenge, we propose a diffusion process that attends to time-dependent spatial frequencies of features with a novel attention mechanism as well as novel noise initialization technique and cross-attention loss. This Fourier-based attention block focuses on features from non-overlapping regions of the generated scene in order to better align the global appearance. Our noise initialization technique incorporates shared noise and low spatial frequency information derived from pixel coordinates and depth maps to induce noise correlations across views. The cross-attention loss further aligns features sharing the same prompt across the scene. Our technique improves SOTA on several quantitative metrics with qualitatively better results when compared to other state-of-the-art approaches for multi-view consistency.

Figures

Figures reproduced from arXiv: 2412.03756 by the authors.

Figure 1
Figure 1. We propose a method that addresses the lack of consistency in multi-view image generation by aligning appearance in non [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed technique. (Left) We initialize noise by sampling Gaussian noise shared across views as well as independent per view. We combined the shared noise with depth or transformed pixel coordinates to obtain Coordinate Noise (Sec. 3.2), which provides a low spatial frequency bias to inform the overall structure of the scene. (Center) We add our Fourier-based Attention (FBA) blocks (blue, see right … view at source ↗
Figure 3
Figure 3. Qualitative comparison for panoramic image generation. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison for multi-view depth-to-image generation. Colored boxes highlight inconsistencies in baselines relative [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Color/texture inconsistencies using CAA (MVDiffusion) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 19 canonical work pages

  1. [1]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation

    work page

  2. [2]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 1

    work page 2023

  3. [3]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. 6

    arXiv 2017

  4. [4]

    Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4), 2023. 2

    work page 2023

  5. [5]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5828–5839, 2017. 6

    work page 2017

  6. [6]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22930–22941, 2023. 1, 3, 4, 9

    work page 2023

  7. [7]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373, 2023. 1, 3

    arXiv 2023

  8. [8]

    Reuse and diffuse: Iterative denoising for text-to-video generation

    Jiaxi Gu, Shicong Wang, Haoyu Zhao, Tianyi Lu, Xing Zhang, Zuxuan Wu, Songcen Xu, Wei Zhang, Yu-Gang Jiang, and Hang Xu. Reuse and diffuse: Iterative denoising for text-to-video generation. arXiv preprint arXiv:2309.03549, 2023. 1

    arXiv 2023

Show all 32 references
  1. [9]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention control. In The Eleventh International Conference on Learning Representations, 2022. 2

    2022

  2. [10]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 6

    2017

  3. [11]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models

    Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. arXiv preprint arXiv:2303.11989, 2023. 1

    2023 arXiv

  4. [12]

    Syncdiffusion: Coherent montage via synchronized joint diffusions

    Yuseung Lee, Kunho Kim, Hyunjin Kim, and Minhyuk Sung. Syncdiffusion: Coherent montage via synchronized joint diffusions. arXiv preprint arXiv:2306.05178, 2023. 1, 2, 6, 7

    2023 arXiv

  5. [13]

    Common diffusion noise schedules and sample steps are flawed

    Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5404–5411, 2024. 1, 3

    2024

  6. [14]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741,

    arXiv

  7. [15]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning. PML...

    2022

  8. [16]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–11, 2023. 5

    2023

  9. [17]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling. arXiv preprint arXiv:2310.15169, 2023. 1, 3

    2023 arXiv

  10. [18]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  11. [19]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022. 1, 2

    2022 arXiv

  12. [20]

    Consisti2v: Enhancing visual consistency for image-to-video generation

    Weiming Ren, Harry Yang, Ge Zhang, Cong Wei, Xinrun Du, Stephen Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324, 2024. 1, 3, 4, 9, 10

    2024 arXiv

  13. [21]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695,

  14. [22]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part I...

    2015

  15. [23]

    Photorealistic text-to-image diffusion models with deep language understand- ing

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gon- tijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understand- ing. Advances in Neural Informa...

    2022

  16. [24]

    Mvdiffusion: Enabling holistic multi-view image generation with correspondence-aware diffusion

    Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. Mvdiffusion: Enabling holistic multi-view image generation with correspondence-aware diffusion. arXiv preprint arXiv:2307.01097, 2023. 1, 3, 4, 5, 6, 7, 9

    2023 arXiv

  17. [25]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 5

    2017

  18. [26]

    Diffusers: State-of-the-art diffusion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers, 2022. 6

    2022

  19. [27]

    Freeinit: Bridging initialization gap in video diffusion models

    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video diffusion models. arXiv preprint arXiv:2312.07537, 2023. 1, 3, 4, 5, 9, 10

    2023 arXiv

  20. [28]

    Freestyle layout-to-image synthesis

    Han Xue, Zhiwu Huang, Qianru Sun, Li Song, and Wenjun Zhang. Freestyle layout-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2

    2023

  21. [29]

    Preserving image properties through initializations in diffusion models

    Jeffrey Zhang, Shao-Yu Chang, Kedan Li, and David Forsyth. Preserving image properties through initializations in diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5242–5250, 2024. 3

    2024

  22. [30]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1, 7

    2023

  23. [31]

    Diffcollage: Parallel generation of large content with diffusion models

    Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming-Yu Liu. Diffcollage: Parallel generation of large content with diffusion models. arXiv preprint arXiv:2303.17076, 2023. 2

    2023 arXiv

  24. [32]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. 6

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.