Pith. sign in

REVIEW 3 major objections 6 minor 59 references

Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fusing identity and text into a single joint embedding and fully fine-tuning SDXL on it lifts ID-preserving generation from about 0.66 to 0.87 face similarity over adapter baselines.

desk verdict A genuinely new joint ID-text conditioning method with strong internal ablations, but the SOTA claim is undercut by evaluation metrics that overlap with training objectives. read the letter →

arxiv 2504.14202 v3 pith:KEHKVZJW submitted 2025-04-19 cs.CV cs.AI

Zichuan Liu , Liming Jiang , Qing Yan , Yumin Jia , Hao Kang , Xin Lu This is my paper · ORCID
classification cs.CVcs.AI
keywords identity-preservingimagesynthesisjointID-textrepresentationmultimodalalignmentFaceCLIPdiffusionmodelsStableXLcontrastivelearningidentityembedding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that identity-preserving portrait generation is better framed as sampling from a forward distribution conditioned on a single unified representation of both the person's identity and the text prompt, rather than as reconstruction through adapters bolted onto a frozen base model. To that end it introduces FaceCLIP, an encoder that fuses face patch features and text patch features into one joint embedding, and a pre-training loss that aligns this embedding to three spaces: CLIP image embeddings, CLIP text embeddings, and ArcFace identity embeddings. The authors then replace SDXL's two text encoders with trained FaceCLIP variants and fine-tune the full diffusion model on this joint condition. They report state-of-the-art face similarity (0.869 on Internal-v1, 0.866 on Unsplash-50), better CLIP scores, lower FID, and higher user preference than InstantID and PuLID-SDXL, with ablation evidence that identity preservation collapses when the identity alignment loss is removed.

What carries the argument

The central object is the FaceCLIP encoder $H_\theta$, which turns a reference face crop and a text prompt into a fused sequence embedding $e \in \mathbb{R}^{b \times L \times d_c}$ via cascaded Feature Fusion blocks built on dual cross-attention plus self-attention. The load-bearing identity is the pre-training objective $L = L_c(e_{c\to t}, e_I) + L_c(e_{c\to r}, e_{r}^{cls}) + L_c(e_{c\to t}, e_t^{cls})$: three contrastive losses that project the fused embedding and align it simultaneously with CLIP image embeddings, ArcFace identity embeddings, and CLIP text embeddings. This alignment is what makes the joint representation usable as a diffusion condition; the paper's ablation attributes the jump from 0.077 to 0.869 face similarity specifically to the ArcFace alignment term, and the text alignment term to the CLIP score gain (0.316 to 0.332). The second pillar is replacing both of SDXL's text encoders with FaceCLIP variants and fine-tuning the entire SDXL backbone, which distinguishes the method from adapter approaches that keep the foundation model fixed.

What would settle it

Compute face similarity with a recognition backbone different from the one used in pre-training, for example a ViT-based face recognizer instead of ArcFace, and run a forced-choice human study on identities and prompts held out from training; if the reported 0.869 advantage shrinks toward the 0.65 baselines on either test, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that identity and text should not be injected separately or via adapters; instead, a single joint representation $e = H_\theta(c_t, c_r)$, produced by fusing identity and text patch embeddings through dual cross-attention, can carry both what the person looks like and what the prompt asks for. Trained with a sum of three contrastive losses—text-image, text-text, and face-identity alignment—this embedding remains discriminative for identity and still transfers well to text-only tasks such as zero-shot ImageNet classification. When SDXL is fully fine-tuned to condition on this joint representation, the paper argues, the model can generate photorealistic portraits that preserve identity and follow prompts more accurately than adapter-injection methods, because the whole generator adapts to the joint condition instead of a small plugin fighting a frozen backbone. The reported numbers support this: face similarity improves from about 0.66 to 0.87, and the ablation shows that removing the identity alignment loss drops face similarity to 0.077.

Load-bearing premise

The load-bearing premise is that the evaluation metrics—ArcFace-based face similarity, CLIP score, and FID—actually measure genuine identity preservation and text adherence; if the high scores come mostly from the generator inverting the same embedding spaces the pre-training aligned to, the state-of-the-art claim would not survive a perceptual or independent test.

Editorial extensions

If this is right

  • If the central claim holds, ID-preserving generation no longer needs adapters or test-time tuning: a single generator fine-tuned on a joint embedding can match or beat plugin-based pipelines, simplifying deployment.
  • The same joint-encoding recipe should transfer to other diffusion backbones by swapping the text encoders for FaceCLIP variants and fine-tuning, since the conditioning is just a fused embedding.
  • The reported ablation implies the identity alignment loss is the necessary ingredient: without it face similarity collapses to 0.077, so any reproduction must keep $L_c(e_{c\to r}, e_r^{cls})$ in the pre-training objective.
  • Because the joint representation still retains text-only capability, with zero-shot ImageNet accuracy close to the OpenCLIP baselines, the trained encoder could serve as a drop-in replacement for the text encoder in ordinary text-to-image generation as well as ID-preserving synthesis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the strongest untested implication is that the method's advantage may be partly metric-circular, since the pre-training aligns the joint embedding to ArcFace, the same space used to compute Face Similarity; a fair test would use an independent face-recognition backbone or human identity judgments on identities held out from training.
  • Editorial inference: the large reported scale, 43 million training samples and more than a hundred GPUs, suggests the gains may depend on compute and dataset size as much as on the architecture; a smaller-scale reproduction with the same losses on a public dataset would separate the algorithm's contribution from scale.
  • Editorial inference: the method should extend naturally to other subject types, such as objects or animals, by swapping the face encoder for a generic object encoder and aligning to a corresponding identity space, since nothing in the fusion or conditioning machinery is face-specific.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FaceCLIP, a multi-modal encoder that fuses an ArcFace-derived identity embedding with CLIP text and image embeddings into a joint representation, and FaceCLIP-SDXL, a pipeline that fully fine-tunes SDXL to condition on this representation. The pre-training objective (Eq. 3) aligns the joint representation with CLIP image, ArcFace identity, and CLIP text embedding spaces, and the authors add a guided image-text dataset to preserve text alignment. Experiments include zero-shot ImageNet classification, t-SNE visualization of identity clusters, ablations over pre-training losses, and comparisons with InstantID and PuLID-SDXL on Internal-v1 and Unsplash-50, reporting Face Similarity, CLIP-T, FID, and a user study. The paper claims state-of-the-art identity preservation, text adherence, and image fidelity.

Significance. The core idea of replacing adapter-based identity injection with a jointly trained identity-text conditioning encoder and fully fine-tuning the base diffusion model is a meaningful alternative to existing approaches, and the large-scale 43M-sample training plus guided-data regularization is an interesting engineering contribution. The paper also provides favorable evidence for the method's internal consistency: the ablation in Table 3 shows that adding identity alignment sharply changes face similarity, the t-SNE plots show identity structure after pre-training, and the guided dataset measurably improves zero-shot classification. However, the headline state-of-the-art claim is currently weakened by a systematic overlap between the spaces used in training objectives and the spaces used in evaluation metrics, which I detail in the major comments. If the authors can validate the results with independent metrics and identity-disjoint evaluation, the contribution would be solid and potentially significant for practical ID-preserving generation.

major comments (3)
  1. [Section 4.1 (Evaluation) and Eq. (3)] The evaluation metrics overlap with the training objectives in a load-bearing way. Face Similarity is the cosine similarity between ArcFace embeddings of reference and generated images, and Eq. (3) explicitly trains FaceCLIP to align its joint representation with the ArcFace embedding er_cls through Lc(ec->r, er_cls). Similarly, CLIP-T is computed as cosine similarity in a CLIP embedding space, while Eq. (3) also aligns ec->t with the CLIP text embedding et_cls and with the CLIP image embedding eI. Because the diffusion model is conditioned on this jointly trained e, the large Face Similarity margins over InstantID and PuLID-SDXL in Table 2 may reflect the model inverting the exact embedding spaces on which it was trained, rather than a generalizable improvement in perceived identity fidelity or text adherence. I am not claiming the method is invalid, but I do claim that the state-of-the-art conclusion is unproven without evaluation in spaces not used during training. Please report Face Similarity with a different face recognition backbone (e.g., AdaFace or FaceNet), compute CLIP-T with a frozen CLIP model that is not used anywhere in FaceCLIP or FaceCLIP-SDXL, and add a human identity-match and text-adherence test with per-criterion breakdowns.
  2. [Section 4.1 (Datasets and Internal-v1)] The Internal-v1 validation set does not appear to be identity-disjoint from the training data. The SPSV dataset is constructed from nine public datasets plus internal datasets, and Internal-v1 is described as an internal validation set with 15 identities and 200 prompts per identity. Since FaceCLIP-SDXL is fully fine-tuned on 43M samples from the same data collection, the high Face Similarity score of 0.869 on Internal-v1 may be partly due to memorization of identities seen during training. The paper states no identity-disjointness check between the training set and Internal-v1. Please report results on identities that are provably absent from the training data, or provide an external identity-disjoint benchmark, and report per-identity variance rather than only the aggregate Face Similarity.
  3. [Section 4.1 (Evaluation, CLIP Score)] The paper does not specify which CLIP model is used to compute CLIP-T. This matters because Section 3.6 states that FaceCLIP-L/14 and FaceCLIP-bigG-14 use OpenAI-CLIP-L-14 and OpenCLIP-bigG-14 backbones, respectively; if CLIP-T is computed with either of these models, the metric evaluates the generated image in exactly the embedding space on which the conditioning encoder was trained. The current wording only says 'CLIP Score is computed as the cosine similarity between the generated image's embedding and the corresponding text prompt's embedding' without naming the model. Please specify the exact CLIP checkpoint and confirm that it is not used as a text or image encoder in FaceCLIP or FaceCLIP-SDXL.
minor comments (6)
  1. [Section 3.6] The sentence 'The diffusion model is then optimized according to Equation 3' appears to be a typo; the diffusion objective is defined in Eq. (2), while Eq. (3) is the pre-training loss for FaceCLIP.
  2. [Section 4.2 and Table 1] The text states that 'FaceCLIP encoders trained with guided data exhibit a notable performance drop,' but Table 1 shows that FaceCLIP-L-14 with guided data achieves 75.3 top-1 accuracy, slightly higher than OpenCLIP-L-14, and both guided variants dramatically outperform their no-guided counterparts (45.8 and 47.2). This sentence appears to refer to a comparison against the original OpenCLIP model for bigG only, or it is a typo; please clarify which comparison is meant.
  3. [Table 3] In the ablation table, L2 and L3 report identical Face Sim scores of 0.869; please report more decimal places or explain why adding the text alignment loss does not change identity similarity.
  4. [Section 4.4.1 (User Study)] The user study compares FaceCLIP-SDXL only with PuLID-SDXL, uses 17 participants, reports a single aggregate preference (68.6% vs. 27.8%), and provides no confidence intervals or per-criterion breakdown for identity similarity, text alignment, and image quality. This does not independently validate the identity-preservation claim.
  5. [Section 3.5 and Section 4.1] The guided-data mixing probability lambda is mentioned in Section 3.5 but never given a value, and several architecture details are missing, including the number of Feature Fusion blocks, the hidden dimension d, and the projection dimensions. Please add these implementation details or refer to released code.
  6. [Table 2] The FID values of 86.9 and 87.8 are high relative to typical FID ranges, and the paper does not state how many generated and reference images were used. Please specify the exact evaluation protocol for FID, including the number of samples.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the ArcFace/CLIP metric overlap is a benchmark-validity limitation, not a construction-level reduction, and the SOTA claim rests on external benchmarks and a user study.

full rationale

The paper's claims are empirical benchmark results rather than first-principles derivations. The only apparent circularity is that Eq. (3) trains the FaceCLIP representation e with contrastive losses that align e to ArcFace (er_cls), CLIP image, and CLIP text embeddings, while the evaluation metrics Face Similarity and CLIP-T measure cosine similarity in those same spaces. This is not a reduction by construction: the diffusion model is trained to generate images conditioned on e, and the metrics are computed on the embeddings of the generated images, a learned mapping that can fail—as the ablation shows, without the ID alignment loss Face Similarity collapses to 0.077. The method is also evaluated on Unsplash-50, an external benchmark not used for training, where the face similarity is 0.866, and by a human user study, providing independent evidence beyond the training objective. The self-citations, such as using CelebV-Text as a training dataset, are not load-bearing for any argumentative step, and the paper imports no uniqueness theorem or ansatz from prior author work. The metric overlap is a legitimate construct-validity and overfitting concern (and the user-study percentages appear to sum to over 100%, a reporting error), but it does not make any derivation equivalent to its inputs by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on: one unreported mixing probability λ; the assumption that CLIP and ArcFace embedding spaces are valid semantic and identity anchors; the assumption that SDXL can be fully fine-tuned on the fused condition without losing quality; and the assumption that the internal 43M-tuple dataset is correctly paired. FaceCLIP itself is a designed artifact rather than an invented entity.

free parameters (1)
  • lambda (guided data mixing probability) = not stated
    Section 3.5 replaces main dataset tuples with guided image-text data with probability λ to preserve text alignment; the reported behavior depends on this balance, and the value is not given.
assumptions (5)
  • domain assumption Pre-trained CLIP and ArcFace embeddings are valid alignment targets for identity and semantics.
    Equation (3) aligns the joint representation to CLIP image, CLIP text, and ArcFace identity spaces, assuming these spaces contain the information needed for generation.
  • domain assumption SDXL can be fully fine-tuned on the new joint condition without catastrophic loss of generative quality.
    Section 3.6 replaces both SDXL text encoders with FaceCLIP and fine-tunes the entire model, relying on the base model retaining image quality and diversity.
  • domain assumption The internal SPSV dataset of 43M tuples has faithful identity-text-image correspondences.
    Section 4.1 assumes InternVL captions correctly describe each portrait and that reference face crops are properly aligned with the target identity.
  • ad hoc to paper Zeroing the face encoder input on guided image-text data preserves text alignment.
    Section 3.5 introduces this regularization as a paper-specific design choice, motivated empirically rather than derived.
  • standard math The CLIP contrastive loss remains effective when aligning projected joint embeddings to three separate spaces.
    Equation (3) reuses the contrastive objective of CLIP [42]; the paper provides no analysis of interactions between the three losses, relying on standard practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis." pith.science (2026). https://pith.science/paper/KEHKVZJW

@misc{pith2026250414202,
  author       = {Pith},
  title        = {Pith review of: Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KEHKVZJW}},
  note         = {Machine review of arXiv:2504.14202}
}
read the original abstract

We propose a novel framework for ID-preserving generation using a multi-modal encoding strategy rather than injecting identity features via adapters into pre-trained models. Our method treats identity and text as a unified conditioning input. To achieve this, we introduce FaceCLIP, a multi-modal encoder that learns a joint embedding space for both identity and textual semantics. Given a reference face and a text prompt, FaceCLIP produces a unified representation that encodes both identity and text, which conditions a base diffusion model to generate images that are identity-consistent and text-aligned. We also present a multi-modal alignment algorithm to train FaceCLIP, using a loss that aligns its joint representation with face, text, and image embedding spaces. We then build FaceCLIP-SDXL, an ID-preserving image synthesis pipeline by integrating FaceCLIP with Stable Diffusion XL (SDXL). Compared to prior methods, FaceCLIP-SDXL enables photorealistic portrait generation with better identity preservation and textual relevance. Extensive experiments demonstrate its quantitative and qualitative superiority.

Figures

Figures reproduced from arXiv: 2504.14202 by the authors.

Figure 1
Figure 1. (a) FaceCLIP architecture and pre-training workflow.The modules labeled in red are freezed [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Visualization of subspaces learned by FaceCLIP-L-14 and FaceCLIP-bigG-14. We visualize [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Images produced by FaceCLIP-SDXL and PuLID-SDXL: The first row are the reference [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 43 canonical work pages

  1. [1]

    Dhariwal, A

    Nichol, A., P. Dhariwal, A. Ramesh, et al. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021

    arXiv 2021

  2. [2]

    Dhariwal, A

    Ramesh, A., P. Dhariwal, A. Nichol, et al. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    arXiv 2022

  3. [3]

    Blattmann, D

    Rombach, R., A. Blattmann, D. Lorenz, et al. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695. 2022

    work page 2022

  4. [4]

    Saharia, C., W. Chan, S. Saxena, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479–36494, 2022

    work page 2022

  5. [5]

    Ruiz, N., Y . Li, V . Jampani, et al. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500–22510. 2023

    work page

  6. [6]

    Zhang, Z

    Wei, Y ., Y . Zhang, Z. Ji, et al. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15943–15953. 2023

    work page 2023

  7. [7]

    Hu, E. J., Y . Shen, P. Wallis, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    work page 2022

  8. [8]

    Alaluf, Y

    Gal, R., Y . Alaluf, Y . Atzmon, et al. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    arXiv 2022

Show all 59 references
  1. [9]

    Zhang, S

    Ye, H., J. Zhang, S. Liu, et al. Ip-adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  2. [10]

    Xiong, Z

    Shi, J., W. Xiong, Z. Lin, et al. Instantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8543–8552. 2024

    2024

  3. [11]

    Guo, Z., Y . Wu, C. Zhuowei, et al. Pulid: Pure and lightning id customization via contrastive alignment. Advances in Neural Information Processing Systems, 37:36777–36804, 2025

    2025

  4. [12]

    Wang, Q., X. Bai, H. Wang, et al. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024

    2024 arXiv

  5. [13]

    He, J., Y . Geng, L. Bo. Uniportrait: A unified framework for identity-preserving single-and multi-human image personalization. arXiv preprint arXiv:2408.05939, 2024

    2024 arXiv

  6. [14]

    Sara Dorfman, R. G. D. C.-R., Dana Cohen Bar. Ip-composer: Semantic composition of visual concepts. arXiv preprint arXiv:2502.13951, 2025

    2025 arXiv

  7. [15]

    Qian, G., K.-C. Wang, O. Patashnik, et al. Omni-id: Holistic identity representation designed for generative tasks. arXiv preprint arXiv:2412.09694, 2024

    2024 arXiv

  8. [16]

    Han, Y ., J. Zhu, K. He, et al. Face-adapter for pre-trained diffusion models with fine-grained id and attribute control. In European Conference on Computer Vision, pages 20–36. Springer, 2024

    2024

  9. [17]

    English, K

    Podell, D., Z. English, K. Lacey, et al. Sdxl: Improving latent diffusion models for high- resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  10. [18]

    Papantoniou, F. P., A. Lattas, S. Moschoglou, et al. Arc2face: A foundation model of human faces. CoRR, 2024

    2024

  11. [19]

    Gabeur, V ., C. Sun, K. Alahari, et al. Multi-modal transformer for video retrieval. InEuropean Conference on Computer Vision, pages 214–229. Springer, 2020

    2020

  12. [20]

    Fei, J., D. Li, Z. Deng, et al. Video-ccam: Enhancing video-language understanding with causal cross-attention masks for short and long videos. arXiv preprint arXiv:2408.14023, 2024

    2024 arXiv

  13. [21]

    Zhu, D., J. Chen, X. Shen, et al. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  14. [22]

    Lin, B., Y . Ye, B. Zhu, et al. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 10

    2023 arXiv

  15. [23]

    Li, C., C. Wong, S. Zhang, et al. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    2023

  16. [24]

    Zhang, D

    Li, B., Y . Zhang, D. Guo, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  17. [25]

    Bai, S., K. Chen, X. Liu, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  18. [26]

    Akhmerov, A. K., A. S. Vasilev, A. V . Vasileva. Research of spatial alignment techniques for multimodal image fusion. In Multimodal Sensing: Technologies and Applications, vol. 11059, pages 309–317. SPIE, 2019

    2019

  19. [27]

    Ahuja, L.-P

    Baltrušaitis, T., C. Ahuja, L.-P. Morency. Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2):423–443, 2018

    2018

  20. [28]

    Selvaraju, A

    Li, J., R. Selvaraju, A. Gotmare, et al. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems , 34:9694–9705, 2021

    2021

  21. [29]

    Barua, A., M. U. Ahmed, S. Begum. A systematic literature review on multimodal machine learning: Applications, challenges, gaps and future directions. Ieee access, 11:14804–14831, 2023

    2023

  22. [30]

    Tian, H., Y . Tao, S. Pouyanfar, et al. Multimodal deep representation learning for video classification. World Wide Web, 22:1325–1341, 2019

    2019

  23. [31]

    Thompson, M

    Shankar, S., L. Thompson, M. Fiterau. Progressive fusion for multimodal integration. arXiv preprint arXiv:2209.00302, 2022

    2022 arXiv

  24. [32]

    Snoek, C. G., M. Worring, A. W. Smeulders. Early versus late fusion in semantic video analysis. In Proceedings of the 13th annual ACM international conference on Multimedia, pages 399–402. 2005

    2005

  25. [33]

    Kulal, A

    Esser, P., S. Kulal, A. Blattmann, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. 2024

    2024

  26. [34]

    Fatras, N

    Tong, A., K. Fatras, N. Malkin, et al. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482, 2023

    2023 arXiv

  27. [35]

    Goldstein, M

    Ma, N., M. Goldstein, M. S. Albergo, et al. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pages 23–40. Springer, 2024

    2024

  28. [36]

    Liu, X., C. Gong, Q. Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022

    2022 arXiv

  29. [37]

    Cheng, X

    Lin, H., X. Cheng, X. Wu, et al. Cat: Cross attention in vision transformer. In 2022 IEEE international conference on multimedia and expo (ICME), pages 1–6. IEEE, 2022

    2022

  30. [38]

    Huang, Y

    Chen, X., L. Huang, Y . Liu, et al. Anydoor: Zero-shot object-level image customization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6593–6602. 2024

    2024

  31. [39]

    Darcet, T

    Oquab, M., T. Darcet, T. Moutakanni, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  32. [40]

    Zhang, Z

    Song, Y ., Z. Zhang, Z. Lin, et al. Imprint: Generative object compositing by learning identity- preserving representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8048–8058. 2024

    2024

  33. [41]

    Deng, J., J. Guo, N. Xue, et al. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699. 2019

    2019

  34. [42]

    Radford, A., J. W. Kim, C. Hallacy, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748–8763. PmLR, 2021

    2021

  35. [43]

    Kornblith, M

    Chen, T., S. Kornblith, M. Norouzi, et al. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PmLR, 2020. 11

    2020

  36. [44]

    Cai, T. T., R. Ma. Theoretical foundations of t-sne for visualizing high-dimensional clustered data. Journal of Machine Learning Research, 23(301):1–54, 2022

    2022

  37. [45]

    Any precision optimizer

  38. [46]

    Cao, Q., L. Shen, W. Xie, et al. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pages 67–74. IEEE, 2018

    2018

  39. [47]

    Zhang, Y

    Guo, Y ., L. Zhang, Y . Hu, et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 87–102. Springer, 2016

    2016

  40. [48]

    Li, Y ., X. Yang, P. Sun, et al. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3207–3216. 2020

    2020

  41. [49]

    Zhu, H., W. Wu, W. Zhu, et al. Celebv-hq: A large-scale video facial attributes dataset. In European conference on computer vision, pages 650–667. Springer, 2022

    2022

  42. [50]

    Laine, T

    Karras, T., S. Laine, T. Aila. A style-based generator architecture for generative adversarial net- works. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410. 2019

    2019

  43. [51]

    Xie, L., X. Wang, H. Zhang, et al. Vfhq: A high-quality dataset and benchmark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 657–666. 2022

    2022

  44. [52]

    Kvanchiani, S

    Kapitanov, A., K. Kvanchiani, S. Kirillova. Easyportrait-face parsing and portrait segmentation dataset. 2023

    2023

  45. [53]

    Yu, J., H. Zhu, L. Jiang, et al. Celebv-text: A large-scale facial text-video dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14805–14814. 2023

    2023

  46. [54]

    Li, S., J. Fu, K. Liu, et al. Cosmicman: A text-to-image foundation model for humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6955–6965. 2024

    2024

  47. [55]

    Chen, Z., J. Wu, W. Wang, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198. 2024

    2024

  48. [56]

    Instantx flux.1-dev ip-adapter page, 2024

    Team, I. Instantx flux.1-dev ip-adapter page, 2024

    2024

  49. [57]

    Ramsauer, T

    Heusel, M., H. Ramsauer, T. Unterthiner, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  50. [58]

    Deng, J., W. Dong, R. Socher, et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  51. [59]

    Learning a parametric embedding by preserving local structure

    Van Der Maaten, L. Learning a parametric embedding by preserving local structure. InArtificial intelligence and statistics, pages 384–391. PMLR, 2009. 12

    2009

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.