Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Graph conditioning sets new FID records for layout and mask synthesis

desk verdict Genuinely new graph-conditioning representation with convincing qualitative control, but the headline FID numbers rest on a confounded comparison. read the letter →

arxiv 2502.01309 v1 pith:53UQ4TZB submitted 2025-02-03 cs.CV

Rupert Menneer , Christos Margadji , Sebastian W. Pattinson This is my paper · ORCID
classification cs.CV
keywords heterogeneousimagegraphsgraph-conditioneddiffusionmagnitude-preservinggraphneuralnetworkControlNetlayout-to-imagesynthesismask-to-imageEDM2scene
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that relational, variable-length conditioning inputs for image synthesis are best handled as a graph, not as text tokens or image concatenation. It builds a Heterogeneous Image Graph (HIG) in which image patches and conditioning objects are nodes, connected by spatial and semantic edges, and processes that graph with a magnitude-preserving graph neural network inserted into the EDM2 diffusion model through a ControlNet adapter. The paper reports that this configuration lowers the previous record Fréchet inception distance (FID) from 15.63 to 8.79 on Visual Genome layout-to-image and from 15.61 to 11.41 on COCO-stuff mask-to-image, at 512×512 resolution. If true, the same graph machinery gives users localised control over object attributes and relationships, including disambiguating overlapping bounding boxes, which earlier layout-conditioning methods handled poorly.

What carries the argument

The load-bearing object is the Heterogeneous Image Graph (HIG): a bipartite structure with unconnected image-patch nodes on one side and conditioning nodes — class labels, bounding boxes, masks, attributes, relationships — on the other, linked by spatial edges determined per task. It is processed by the HIGConv operator, a magnitude-preserving message-passing layer inspired by GraphSAGE that averages neighbourhood features scaled by the square root of the neighbourhood size, applies forced weight normalisation from EDM2, and sums across meta-paths with inverse-square-root scaling so hidden-state magnitudes remain bounded. The HIG branch is attached to a frozen EDM2 U-Net through a ControlNet-style zero-gain integration that uses magnitude-preserving summation instead of plain addition; the whole mechanism lets the denoiser attend to arbitrary graph conditions repeatedly during sampling.

What would settle it

Run the same frozen EDM2 backbone at 512×512 with the same auto-guidance strength, feeding masks or bounding boxes through a non-graph ControlNet adapter (for example by concatenating a layout tensor into the encoder), and compare FID on the COCO-stuff and Visual Genome validation splits; if the non-graph adapter matches 11.41 and 8.79, the HIG graph structure is not what drives the reported improvement.

Watch

Extended reading notes

Core claim

The central claim is that one heterogeneous graph representation can serve as a universal conditioning channel for diffusion image synthesis. The HIG keeps image features as unconnected patch nodes and connects them to conditioning nodes — objects, masks, attributes, relationships — through task-specific spatial edges; a message-passing GNN then exchanges information between image and conditioning nodes inside the denoising network. The authors supplement this with a magnitude-preserving graph convolution whose sums and concatenations are normalized so feature magnitudes do not drift, and they integrate the graph branch into a frozen EDM2 backbone with a ControlNet-style trainable encoder, using the unconditional EDM2 checkpoint for auto-guidance. On COCO-stuff and Visual Genome, the authors report FID improvements over the previous layout-to-image and mask-to-image baselines while using less filtered validation sets, and they demonstrate seed-preserving edits of object position, size, colour, and inter-object ordering.

Load-bearing premise

The headline 'improves upon SOTA' rests on comparing 512×512 generations from an ImageNet-pretrained EDM2 backbone with auto-guidance 1.8 against the 256×256 LayoutDiffusion baseline; if resolution or backbone alone accounts for the FID drop, the graph representation is not the cause.

Editorial extensions

If this is right

  • Conditioning inputs of varying length and heterogeneous modalities can be fused in one forward pass without quadratic attention over boxes, since the GNN aggregates only local neighbourhoods.
  • Same-seed edits of size, position, colour, and object relationships become possible by editing graph nodes or edges, making the representation usable for image editing.
  • Mask conditioning improves YOLOScore and controllability relative to box-only conditioning, at the cost of a lower diversity score.
  • The method tolerates less aggressively filtered validation sets than earlier scene-graph work, suggesting the graph representation generalises across label distributions.
  • Any conditioning signal expressible as nodes and edges, including CLIP-encoded captions, can be fed through the same graph-conditioning block.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the reported gains mix a higher resolution (512 versus 256), a stronger ImageNet-pretrained backbone (EDM2), and auto-guidance strength 1.8 with the graph representation, the contribution of the graph itself is not separately measured; a control comparing the same backbone with mask or layout concatenation would isolate it.
  • Beyond the paper: the magnitude-preserving message-passing recipe may transfer to other ControlNet-style conditioning tasks, since the ablation shows plain addition produces NaNs while the normalized sum trains stably.
  • Beyond the paper: the HIG's unconnected image nodes suggest a general recipe for attaching sparse conditioning to latent diffusion models without modifying the base U-Net, which could extend to non-image structured data such as weather or manufacturing sensor graphs.
  • Beyond the paper: the observation that standard FID favours the smaller model while DINOv2-FID favours the larger, under-trained XXL model implies the choice of evaluation metric could reverse the ranking in future head-to-head comparisons.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Heterogeneous Image Graphs (HIG), a graph representation that connects image patch nodes to conditioning nodes (e.g., bounding boxes, segmentation masks, attributes, relationships), and a magnitude-preserving GNN (MP-GNN) inserted as a ControlNet-style adapter into the EDM2 diffusion backbone. The authors report FID improvements over LayoutDiffusion on COCO-stuff and Visual Genome layout/mask-to-image tasks, and demonstrate controllability in qualitative experiments such as local attribute edits and disambiguation of overlapping bounding boxes.

Significance. If the quantitative claims are supported, the HIG representation is a valuable step toward flexible, relation-aware conditioning for diffusion models. The architecture is lightweight (7.0-18.9M GNN parameters), reuses a strong pretrained backbone, and the qualitative results show fine-grained control that current layout-conditioning methods do not. The paper would be strengthened by corroborating these claims with fair, matched baselines; the experimental comparison as presented does not yet establish the claimed state of the art.

major comments (4)
  1. [Section 4.3, Table 1] The headline SOTA claim is not supported by the comparison in Table 1. All HIG rows are generated at 512×512 with an ImageNet-pretrained EDM2 backbone and auto-guidance strength 1.8, whereas LayoutDiffusion is evaluated at 128×128 or 256×256 without auto-guidance. FID values computed at different resolutions are not directly comparable, and auto-guidance alone is known to shift FID. A matched baseline — e.g., the same EDM2 backbone with a simple mask- or box-concatenation ControlNet at 512×512 — is required to attribute the observed gains to the HIG/MP-GNN design.
  2. [Section 4.3, Table 1 (COCO-stuff rows)] The COCO-stuff task is mismatched with the only baseline provided. The paper's abstract and Section 1 frame the 11.41 FID as a mask-to-image result, but LayoutDiffusion is a layout-to-image model conditioned on bounding boxes, not masks. Masks provide object boundaries "for free," so a trivial mask-conditioned baseline would be expected to outperform a box-conditioned model. Without a mask-conditioned baseline, the COCO FID gap cannot be attributed to the graph representation.
  3. [Sections 4.1-4.3; Appendix C, Table 3] The FID results carry no error bars or multiple-seed variance, despite the paper stating that generations use 5 samples per condition. Furthermore, the models are explicitly under-trained: Section 4.1 notes the models "do not witness convergence" after 12 GPU-days, and Table 2 reports HIG-Medium at 18.34 FID on a 12M-image run — far above the Table 1 value. The absolute numbers in Table 1 are therefore fragile point estimates, and the gap to LayoutDiffusion may partly reflect backbone, guidance, or resolution rather than the HIG method.
  4. [Appendix A, Eqs. (9)-(10); Section 3.3, Eq. (3)] The magnitude-preserving guarantee rests on an independence assumption that is not satisfied by the actual graph constructions. The proof assumes random zero-mean unit vectors, but in the HIG, image nodes inside a bounding box are highly correlated and are aggregated into the same conditioning node's neighborhood. The paper acknowledges the assumption fails for "identical features attached to the same node," but the broader failure mode for spatially adjacent latent patches is not addressed. Since the ablation in Table 2 shows minimal difference between MP-GNN and PixNorm (18.34 vs 18.56), the formal guarantee is not empirically load-bearing, but the claim as stated in Section 3.3 is misleading and should be tempered or replaced with a condition that holds for the data.
minor comments (5)
  1. [Section 4.3] The sentence "For a fair comparison we present results on generated samples with and without mask inputs" is misleading because it refers only to HIG's own mask ablation, not to a fair comparison with LayoutDiffusion; please rephrase.
  2. [Table 1] FID_DINOv2 is not reported for LayoutDiffusion, so the claim of "superior performance in all metrics" is not supported for that metric, since the comparison is missing.
  3. [Throughout] The word "heterogenous" is consistently misspelled; it should be "heterogeneous." There are also minor typos such as "similiar" in Section 3.3.
  4. [Section 4.3] The reference to "Appendix 8" should be "Appendix Figure 8" for the guidance-strength plot.
  5. [Section 4.1, Implementation Details] Please specify the actual training iteration counts or image counts for the Table 1 models, since the text says "12 A100 GPU days" and Appendix C reports "Duration 40M" and "Training time 3.0 days," which is ambiguous to readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical benchmark results and an architectural proposal, none of which reduce to their inputs by construction.

full rationale

The paper makes two kinds of claims: a new conditioning architecture (HIG with a magnitude-preserving GNN integrated into EDM2 via a ControlNet-style adapter) and empirical SOTA claims on COCO-Stuff and Visual Genome FID. Neither reduces to its own inputs by construction. The HIG representation is defined from masks, boxes, attributes, and relationships, while the reported FID values are computed on validation splits against external baselines; no parameter is fitted to the target FID and then renamed a prediction. The magnitude-preservation argument in Appendix A explicitly assumes 'independent random unit vectors' and 'standardised' inputs, and the paper itself notes the condition fails when identical features are attached to the same node; this is a stated, possibly violated assumption rather than a circular definition. The SOTA comparison is confounded—512px EDM2-based HIG with auto-guidance 1.8 versus 256px LayoutDiffusion, and mask-conditioned runs versus a box-only baseline—but that is a benchmark-validity problem, not circular reasoning: the compared numbers come from independent training runs, not from the definition of the method. The authors also disclose under-training ('the original model was pre-trained for 291 GPU days, whereas ours was trained for only 12') and non-convergence, which are limitations rather than self-referential proofs. There are no load-bearing self-citations by the present authors. Therefore no circular step is exhibited, and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small set of hand-chosen hyperparameters (guidance strength, mask dropout, VG filtering thresholds) and on three domain assumptions: that pretrained ImageNet EDM2 transfers to COCO/VG, that CLIP encodes relational semantics adequately, and that the random-vector independence assumption justifies magnitude preservation. No new physical or ontological entities are introduced.

free parameters (3)
  • auto-guidance strength = 1.8
    Selected to optimize FID_DINOv2; Appendix Figure 8 shows the FID optimum is lower, so the headline FID numbers are tied to this choice.
  • mask node dropout probability = 0.5
    Chosen by hand in Section 4.1 to avoid over-reliance on mask inputs; affects the controllability and diversity tradeoff.
  • Visual Genome filtering thresholds = 250 for attributes/relationships, 1000 for objects
    Chosen to reduce vocabulary size; the paper notes this is less strict than prior work (500/2000), which changes task difficulty and comparability.
assumptions (4)
  • ad hoc to paper Neighborhood features aggregated in Equation 3 behave as independent random zero-mean unit vectors, so summing and dividing by sqrt(|N|) preserves expected norm.
    Appendix A proves expected norm only under independence and zero-mean assumptions; actual learned GNN features are deterministic and correlated, so the stated 'guaranteed' magnitude preservation is not established.
  • domain assumption The pretrained EDM2 model trained on ImageNet classes can be adapted to natural image datasets by unfreezing only the embedding network.
    Section 3.3 and Section 4.1: COCO/VG are not ImageNet, and the paper observes that the adaptation works but does not ablate this transfer separately.
  • domain assumption CLIP-ViT-Large embeddings of class labels, attributes, and relationship words carry enough semantic information to serve as conditioning features.
    Section 4.1: all text-based conditioning is pre-encoded with CLIP; there is no dedicated evaluation of CLIP encoding fidelity for spatial or relational semantics.
  • standard math The standard score-matching and probability-flow ODE framework of EDM2 applies unchanged when the ControlNet-modified network is used.
    The paper explicitly defers diffusion theory to prior work and uses the EDM2 training recipe, so this is a standard background assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis." pith.science (2026). https://pith.science/paper/53UQ4TZB

@misc{pith2026250201309,
  author       = {Pith},
  title        = {Pith review of: Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/53UQ4TZB}},
  note         = {Machine review of arXiv:2502.01309}
}
read the original abstract

We introduce a novel method for conditioning diffusion-based image synthesis models with heterogeneous graph data. Existing approaches typically incorporate conditioning variables directly into model architectures, either through cross-attention layers that attend to text latents or image concatenation that spatially restrict generation. However, these methods struggle to handle complex scenarios involving diverse, relational conditioning variables, which are more naturally represented as unstructured graphs. This paper presents Heterogeneous Image Graphs (HIG), a novel representation that models conditioning variables and target images as two interconnected graphs, enabling efficient handling of variable-length conditioning inputs and their relationships. We also propose a magnitude-preserving GNN that integrates the HIG into the existing EDM2 diffusion model using a ControlNet approach. Our approach improves upon the SOTA on a variety of conditioning inputs for the COCO-stuff and Visual Genome datasets, and showcases the ability to condition on graph attributes and relationships represented by edges in the HIG.

Figures

Figures reproduced from arXiv: 2502.01309 by the authors.

Figure 1
Figure 1. (a) Our representation enables flexible conditioning for graph-to-image generation by modeling objects, attributes, and relationships as a graph. This integrates with a pixel grid, where pixels act as nodes connected to objects, defining spatial relation￾ships. (b) Modifying conditions at the node and or edge level enables precise semantic control over the generation process. 2005). Typically, the image processing a… view at source ↗
Figure 2
Figure 2. (a) Overview of the proposed architecture. The HIG is encoded into a latent representation through a MP-GNN which is then used as a condition cf in a ControlNet. (b) Details of the MP-GNN module. Note: HMP is shorthand for heterogenous magnitude preserving operations applied across all nodes. data opt for complex multi-stage training procedures such as masked contrastive pre-training using graph triplets (Yang et al… view at source ↗
Figure 4
Figure 4. Generations from the same seed demonstrate HIG’s abil￾ity to control size, quantity and position of objects. approach. Specifically, HIG with bounding boxes gener￾ates highly realistic images, outperforming previous SOTA in terms of detail and coherence. Furthermore, HIG with masks excels in accurately preserving the original sub￾structures of the image, as explicitly guided by the given masks. This capability makes… view at source ↗
Figures from the paper (14 more)
Figure 5
Figure 5. Figure 5: HIG enables precise, localized control over semantic conditions, including attributes, objects, and their relationships [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: We show HIG can effectively generate consistent images, adhering to underlying mask and semantic conditions. In this case we show ‘HIG’ written in trees, food, windows and kites. tions, such as a cat-shaped broccoli, positioned beneath a hat. While object-dependent spa…
Figure 7
Figure 7. Figure 7: Selected generated samples from HIG using COCO validation conditions. 3 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: FID and FIDDINOv2 vs guidance. To understand the relationship between auto-guidance strength and FID values we generate 5K images for coco validation (i.e. one seed). The relationship we witness is similiar to that reported in the original (Karras et al., 2024b) work, …
Figure 9
Figure 9. Figure 9: Diverse HIG samples using mask and bounding boxes from COCO validation set. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Diverse HIG samples using only bounding boxes from the COCO validation set. We naturally observe increased diversity in images when compared to [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Head-to-head comparisons between generations from HIG-Medium and HIG-XXL. There is a notable enhancement in realism in the latter, which exhibit richer color depth and more convincing surface textures [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Selected generations using a single bounding box with an animal class label. 7 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: We showcase the ability of the HIG to disambiguate overlapping bounding boxes. In this figure we show bear, horse, elephant and cow bounding boxes overlapped with a tree bounding box, with the relationship ‘in front’ or ’behind’. We also show the bounding box when the…
Figure 14
Figure 14. Figure 14: We showcase the ability to locally edit the attribute of an object i.e. the colour of a specific animal in an image. 9 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: We showcase the ability to locally edit the attribute of an object i.e. the colour of a specific bus or flower in a generation. 10 [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]
Figure 16
Figure 16. Figure 16: Generations with different colour attributes. 11 [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]
Figure 17
Figure 17. Figure 17: Editing a scenes layout. We observe that when generating images with the same seed that object semantics stay relatively consistent as we add or remove objects from a scene. 12 [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: We showcase the ability to change the size and position of an object in a scene. 13 [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 5 canonical work pages

  1. [3]

    Semantic Image Manipulation Using Scene Graphs

    URL http:// arxiv.org/abs/2004.03677. arXiv:2004.03677 [cs]. Dhariwal, P. and Nichol, A. Diffusion Models Beat GANs on Image Synthesis, June

    work page Pith review arXiv 2004

  2. [7]

    arXiv:1704.01212 [cs]

    URL http://arxiv.org/abs/ 1704.01212. arXiv:1704.01212 [cs]. Hamilton, W. L., Ying, R., and Leskovec, J. Induc- tive Representation Learning on Large Graphs, Septem- ber

    arXiv

  3. [10]

    URL https://arxiv.org/abs/1706.08500. Ho, J. and Salimans, T. Classifier-Free Diffusion Guidance, July

    arXiv

  4. [11]

    org/abs/2006.11239

    URLhttp://arxiv. org/abs/2006.11239. arXiv:2006.11239 [cs, stat]. Hyv¨arinen, A. Estimation of Non-Normalized Statistical Models by Score Matching. J. Mach. Learn. Res. , 6: 695–709, December

    arXiv 2006

  5. [12]

    org/abs/1804.01622

    URL http://arxiv. org/abs/1804.01622. arXiv:1804.01622 [cs]. Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the Design Space of Diffusion-Based Generative Mod- els, October

    arXiv

  6. [13]

    arXiv:2206.00364 [cs, stat]

    URL http://arxiv.org/abs/ 2206.00364. arXiv:2206.00364 [cs, stat]. Karras, T., Aittala, M., Kynk¨a¨anniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding a Diffusion Model with a Bad Version of Itself, June 2024a. URL http://arxiv. org/abs/2406.02507. arXiv:2406.02507 [cs, stat]. Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., and Laine...

    arXiv

  7. [14]

    URL http://arxiv.org/abs/1609. 02907. arXiv:1609.02907 [cs, stat]. Krzywda, M., Łukasik, S., and Gandomi, A. H. Graph Neural Networks in Computer Vision – Architectures, Datasets and Common Approaches. In 2022 Interna- tional Joint Conference on Neural Networks (IJCNN) , pp. 1–10, July

    arXiv 2022

  8. [15]

    doi: 10.1109/IJCNN55064.2022. 9892658. URL http://arxiv.org/abs/2212. 10207. arXiv:2212.10207 [cs]. Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton- Rosen, Z., Hu, W., Merose, A., Hoyer, S., Holland, G., Vinyals, O., Stott, J., Pritzel, A., Mohamed, S., and Battaglia, P. GraphCast: Learni...

    arXiv 2022

Show all 41 references
  1. [16]

    Li, M., Yang, T., Kuang, H., Wu, J., Wang, Z., Xiao, X., and Chen, C

    URL https://arxiv.org/abs/2212.12794v2. Li, M., Yang, T., Kuang, H., Wu, J., Wang, Z., Xiao, X., and Chen, C. ControlNet++: Improving Condi- tional Controls with Efficient Consistency Feedback, July

    arXiv

  2. [17]

    URL http://arxiv.org/abs/2404. 07987. arXiv:2404.07987 [cs]. Li, Z., Wu, J., Koh, I., Tang, Y ., and Sun, L. Image synthesis from layout with locality-aware mask adaption. In IEEE International Conference on Computer Vision (ICCV), pp. 13819–13828. IEEE,

    arXiv

  3. [18]

    doi: 10.1109/TGRS

    ISSN 1558-0644. doi: 10.1109/TGRS. 2020.3037361. URL https://ieeexplore.ieee. org/document/9268479. Conference Name: IEEE Transactions on Geoscience and Remote Sensing. Mittal, G., Agrawal, S., Agarwal, A., Mehta, S., and Mar- wah, T. Interactive Image Generation Using Scene G...

    2020

  4. [20]

    URL http://arxiv.org/abs/2304. 07193. arXiv:2304.07193 [cs]. Peebles, W. and Xie, S. Scalable Diffusion Models with Transformers, March

    arXiv

  5. [21]

    org/abs/2212.09748

    URL http://arxiv. org/abs/2212.09748. arXiv:2212.09748 [cs]. Podell, D., English, Z., Lacey, K., Blattmann, A., Dock- horn, T., M¨uller, J., Penna, J., and Rombach, R. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis, July

    arXiv

  6. [22]

    org/abs/2307.01952

    URL http://arxiv. org/abs/2307.01952. arXiv:2307.01952 [cs]. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision,

    arXiv

  7. [23]

    Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M

    URL https://arxiv.org/abs/2103.00020. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical Text-Conditional Image Generation with CLIP Latents, April

    arXiv

  8. [24]

    org/abs/2204.06125

    URL http://arxiv. org/abs/2204.06125. arXiv:2204.06125 [cs]. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models, April

    arXiv

  9. [25]

    org/abs/2112.10752

    URL http://arxiv. org/abs/2112.10752. arXiv:2112.10752 [cs]. Ronneberger, O., Fischer, P., and Brox, T. U-Net: Convo- lutional Networks for Biomedical Image Segmentation, May

    arXiv

  10. [27]

    arXiv:2104.07636 [cs, eess]

    URL http://arxiv.org/ abs/2104.07636. arXiv:2104.07636 [cs, eess]. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mah- davi, S. S., Lopes, R. G., Salimans, T., Ho, J., Fleet, D. J., and Norouzi, M. Photorealistic Text-to-I...

    arXiv

  11. [28]

    URL http://arxiv.org/abs/2205. 11487. arXiv:2205.11487 [cs]. Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S. Deep Unsupervised Learn- ing using Nonequilibrium Thermodynamics, Novem- ber

    arXiv

  12. [29]

    URL http://arxiv.org/abs/1503. 03585. arXiv:1503.03585 [cond-mat, q-bio, stat]. 10 Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis Song, Y ., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-Based Generative Mod- eling throu...

    arXiv

  13. [30]

    URL http://arxiv.org/abs/2011. 13456. arXiv:2011.13456 [cs, stat]. Stein, G., Cresswell, J. C., Hosseinzadeh, R., Sui, Y ., Ross, B. L., Villecroze, V ., Liu, Z., Caterini, A. L., Taylor, J. E. T., and Loaiza-Ganem, G. Exposing flaws of genera- tive model evaluation metrics an...

    2011 arXiv

  14. [31]

    org/abs/2306.04675

    URL http://arxiv. org/abs/2306.04675. arXiv:2306.04675 [cs]. Szegedy, C., Vanhoucke, V ., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vi- sion,

    arXiv

  15. [32]

    URL https://arxiv.org/abs/1512. 00567. Tarasiewicz, T., Nalepa, J., and Kawulok, M. A Graph Neural Network For Multiple-Image Super- Resolution. In 2021 IEEE International Confer- ence on Image Processing (ICIP) , pp. 1824–1828, September

    2021

  16. [33]

    doi: 10.1109/ICIP42928.2021. 9506070. URL https://ieeexplore.ieee. org/document/9506070. ISSN: 2381-8549. Tian, Y ., Chen, H., Xu, C., and Wang, Y . Image Processing GNN: Breaking Rigidity in Super-Resolution. Veliˇckovi´c, P., Cucurull, G., Casanova, A., Romero, A., Li`o, P.,...

    2021

  17. [34]

    URL http://arxiv.org/abs/1710. 10903. arXiv:1710.10903 [cs, stat]. Wan, S., Gong, C., Zhong, P., Du, B., Zhang, L., and Yang, J. Multi-scale Dynamic Graph Convolu- tional Network for Hyperspectral Image Classification, May

    arXiv

  18. [35]

    URL http://arxiv.org/abs/1905. 06133. arXiv:1905.06133 [cs, eess, stat]. Xu, D., Zhu, Y ., Choy, C. B., and Fei-Fei, L. Scene Graph Generation by Iterative Message Passing, April

    1905 arXiv

  19. [36]

    URL http://arxiv.org/abs/1701. 02426. arXiv:1701.02426 [cs]. Yang, J., Lu, J., Lee, S., Batra, D., and Parikh, D. Graph R-CNN for Scene Graph Generation, Au- gust

    arXiv

  20. [37]

    URL http://arxiv.org/abs/1808. 00191. arXiv:1808.00191 [cs]. Yang, L., Huang, Z., Song, Y ., Hong, S., Li, G., Zhang, W., Cui, B., Ghanem, B., and Yang, M.-H. Diffusion-Based Scene Graph to Image Generation with Masked Con- trastive Pre-Training, November

    arXiv

  21. [38]

    arXiv:2211.11138 [cs]

    URL http:// arxiv.org/abs/2211.11138. arXiv:2211.11138 [cs]. Zhang, L., Rao, A., and Agrawala, M. Adding Conditional Control to Text-to-Image Diffusion Models, Novem- ber

    arXiv

  22. [39]

    URL http://arxiv.org/abs/2302. 05543. arXiv:2302.05543 [cs]. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR,

    arXiv

  23. [40]

    arXiv:2303.17189 [cs]

    URL http:// arxiv.org/abs/2303.17189. arXiv:2303.17189 [cs]. 11 Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis Appendix A. Sum of Random Unit Vectors First, recall that for random zero-mean vectors, their expected Euclidean normE[∥a∥2] = E[a2 1 + a2 2...

    arXiv

  24. [41]

    Training details for HIGnn diffusion models. Model details M XXL Number of GPUs 4 4 Minibatch size 256 256 Duration 40M 15M Channel multiplier 256 448 Number of HIG blocks 4 4 Dropout probability 10% 10% Learning rate max (αref ) 0.0045 0.003 Learning rate decay (tref ) 10000 ...

  25. [2015]

    URL http://arxiv.org/abs/1505. 04597. arXiv:1505.04597 [cs]. Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., and Norouzi, M. Image Super-Resolution via Iterative Refinement, June

    arXiv

  26. [2017]

    arXiv:1606.09375 [cs, stat]

    URL http:// arxiv.org/abs/1606.09375. arXiv:1606.09375 [cs, stat]. Dhamo, H., Farshad, A., Laina, I., Navab, N., Hager, G. D., Tombari, F., and Rupprecht, C. Semantic Image Manipu- lation Using Scene Graphs, April

    arXiv

  27. [2018]

    URL http://arxiv.org/abs/1706. 02216. arXiv:1706.02216 [cs, stat]. Han, K., Wang, Y ., Guo, J., Tang, Y ., and Wu, E. Vi- sion GNN: An Image is Worth Graph of Nodes, Novem- ber

    arXiv

  28. [2019]

    arXiv:1905.03743 [cs]

    URL http://arxiv.org/abs/ 1905.03743. arXiv:1905.03743 [cs]. Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y ., Li, S.-W., Misra, I., R...

    1905 arXiv

  29. [2020]

    Defferrard, M., Bresson, X., and Vandergheynst, P

    URL https://arxiv.org/abs/2004.10934. Defferrard, M., Bresson, X., and Vandergheynst, P. Con- volutional Neural Networks on Graphs with Fast Local- ized Spectral Filtering, February

    2004 arXiv

  30. [2021]

    org/abs/2105.05233

    URL http://arxiv. org/abs/2105.05233. arXiv:2105.05233 [cs, stat]. Eiras, F., Petrov, A., Vidgen, B., Schroeder, C., Pizzati, F., Elkins, K., Mukhopadhyay, S., Bibi, A., Purewal, A., Botos, C., Steibel, F., Keshtkar, F., Barez, F., Smith, G., Guadagni, G., Chun, J., Cabot, J.,...

    arXiv

  31. [2022]

    URL http://arxiv.org/abs/2206. 00272. arXiv:2206.00272 [cs]. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    arXiv

  32. [2023]

    arXiv:2304.14573 [cs]

    URL http:// arxiv.org/abs/2304.14573. arXiv:2304.14573 [cs]. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural Message Passing for Quantum Chem- istry, June

    arXiv

  33. [2024]

    org/abs/2405.08597

    URL https://arxiv. org/abs/2405.08597. Farshad, A., Yeganeh, Y ., Chi, Y ., Shen, C., Ommer, B., and Navab, N. SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis, April

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.