{"id":"54e1f902-43ea-4315-b60f-afbd94ba9c77","arxiv_id":"2505.05081","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper reports a per-identity fine-tuning method using StyleGAN's layered face codes as visual prompts, and it claims improved identity preservation and text alignment over previous baselines.","lead":"PIDiff is a method for keeping a specific person's face recognizable in new AI-generated pictures while changing the pose, clothing, or scene. It combines a layered face representation from StyleGAN with a fine-tuned diffusion model, and the paper reports better identity preservation than several earlier personalization methods on its own benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative superiority is not yet demonstrated: Table 1 is weakened by hyperparameters selected on the evaluation set (λ=0.4, six training images) and by a nonstandard, method-dependent reference-scoring rule in Sec. 3.3.1.","rationale":"The paper's headline is empirical: PIDiff outperforms state-of-the-art methods. The reader's weakest_assumption focused on e4e disentanglement, which is the key mechanism risk; I partially agree with that. However, the more load-bearing issue is that the quantitative evidence itself is not comparable or statistically grounded. The 'most similar training image' rule changes the meaning of ID for reference-free baselines, and λ and image-count selection are performed on the same benchmark that later becomes Table 1. The absence of error bars makes the tiny CLIP-T and LPIPS gaps uninterpretable. A held-out rerun with fixed references and repeated seeds would settle this. The concern does not disprove the method: the qualitative results and attention-map evidence are suggestive, and the underlying W+ and SCA ideas are plausible. Therefore I do not move the verdict; the reader's CONDITIONAL verdict already reflects the evaluation weaknesses, and my concern reinforces it rather than changing it.","tokens_in":12935,"tokens_out":8713,"duration_ms":91444,"concrete_test":"Re-run the Sec. 3.3 comparison on a held-out set of prompts (and ideally held-out identities) that were not used to choose λ, the number of training images, or the visual-drop probability. Use the same fixed reference image per identity for every method, including reference-free baselines, instead of selecting the most similar training face. Compute ID, LPIPS, and CLIP-T per prompt with at least three seeds and report mean±std. If PIDiff's ID/CLIP-T advantage over W+ Adapter and IP-Adapter is no longer larger than the run-to-run noise, the Table 1 claim of state-of-the-art outperformance is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest evidence for the central claim is Table 1, but the protocol behind it does not support the cross-method comparison as reported. First, in Sec. 3.3.1, for methods without an inference-time reference image (Textual Inversion, Custom Diffusion, DreamBooth), the ID score is computed against 'the most similar facial image from the training images' rather than a fixed reference image. This is a per-sample max over a 10-image gallery, which is a different metric from the fixed-reference protocol used for IP-Adapter, W+ Adapter, and PIDiff, so the rows are not directly comparable. Second, hyperparameters that directly affect the reported scores are chosen by looking at those same scores: λ=0.4 is selected in Sec. 3.4.4 because it maximizes ID/CLIP-T on the same 12 prompts × 27 identities used in Table 1, and the training-image count of six is selected in Sec. 3.4.2 from the same benchmark. No held-out split is described. With λ=0.4, PIDiff's CLIP-T is 0.1938 versus W+ Adapter's 0.1935, and its LPIPS gap to IP-Adapter is 0.0011; without error bars or multiple seeds, these margins are within plausible run-to-run noise. The e4e-disentanglement assumption is a real mechanism risk, but even if W+ were perfectly disentangled, the comparative claim is not established if the numbers are produced by a tuned, nonstandard evaluation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PIDiff, a fine-tuning-based method for personalized identity text-to-image generation. The method maps a cropped and aligned face image to the StyleGAN W+ space using the e4e encoder, transforms the resulting w+ vector into visual tokens through a mapping network, and injects these tokens into Stable Diffusion through a newly designed Style Cross-Attention (SCA) block placed after each text cross-attention block. Only the SCA projection matrices and the mapping network are trainable during a short per-identity fine-tuning stage. The paper also introduces a small dataset of 27 identities with 10 images each and reports that PIDiff achieves the highest identity similarity (ID 0.3112) and the best or tied text alignment (CLIP-T 0.1938) on this dataset, along with qualitative results and an ablation study that selects the number of training images and the SCA weight λ. The central claim is that PIDiff outperforms state-of-the-art methods while better preserving identity and maintaining text-image consistency.","tokens_in":13262,"tokens_out":4276,"duration_ms":42507,"significance":"If the central claim holds, the paper makes a useful contribution by demonstrating that a StyleGAN W+ visual prompt can be integrated into a diffusion model for identity-preserving generation, and that a serial cross-attention structure after text cross-attention can improve visual-prompt localization. The idea of limiting trainable parameters to the SCA projections and mapping network is sensible and aligns with recent parameter-efficient fine-tuning practice. The paper also makes a positive effort to construct a dataset with balanced demographic categories. The strengths are the architectural proposal, the attention-map analysis, and the style-editing demonstration via coarse-to-fine w+ vector concatenation. However, the quantitative evidence as currently presented is not sufficient to establish the claimed superiority over prior methods, for the reasons detailed in the major comments.","major_comments":[{"comment":"The ID evaluation protocol is not comparable across the rows of Table 1. For methods that do not take a reference image at inference time (Textual Inversion, Custom Diffusion, DreamBooth, and possibly BLIP-Diffusion and VICO), the paper states that it selects the most similar facial image from the training images to compute the evaluation metrics, whereas for IP-Adapter, W+ Adapter, and PIDiff the ID is computed against a fixed reference image. This is a per-sample maximum over a 10-image gallery for some methods and a fixed-reference protocol for others, which can inflate ID scores for the reference-free methods and makes the cross-method comparison misleading. The authors should use the same fixed-reference scoring rule for all methods, or otherwise justify why the per-sample max is appropriate and show that the conclusions do not change under a uniform protocol.","section":"Sec. 3.3.1, Table 1"},{"comment":"The reported quantitative superiority is partly circular because the two key hyperparameters are selected on the same benchmark that is then used for the final comparison. Section 3.4.4 states that λ=0.4 is chosen because it maximizes text-image semantic consistency and identity preservation, presumably on the same 12 prompts and 27 identities used in Table 1, and Section 3.4.2 selects six training images from the same Table 2 results. No held-out split or nested selection procedure is described. The authors should evaluate on a disjoint held-out set of identities and prompts, or at least report the selection procedure and show that the final ranking is stable under cross-validation.","section":"Sec. 3.4.4 and Table 2"},{"comment":"Table 1 reports no error bars, confidence intervals, or multiple-seed results. Several of the decisive margins are very small: the CLIP-T difference between PIDiff (0.1938) and W+ Adapter (0.1935) is 0.0003, and the LPIPS difference between PIDiff (0.5936) and IP-Adapter (0.5947) is 0.0011. Given the stochasticity of diffusion sampling and the small evaluation set, these differences are within plausible run-to-run noise. The authors should run each method and their own method with multiple random seeds and report mean ± std, and should state whether the reported numbers are averages or single runs.","section":"Table 1"},{"comment":"The central mechanism claim that cropping and e4e inversion into W+ spaces disentangles identity information from background and other scene content is not quantitatively verified. The paper only provides attention-map visualizations in Fig. 3, which are suggestive but not a quantitative test. This assumption is load-bearing for the claimed advantage over W+ Adapter, because if scene information survives in W+ the visual prompt could still interfere with backgrounds and the SCA localization would fail. A concrete test would be, for example, measuring whether the generated background changes when the same identity is presented in different backgrounds, or comparing attention maps and output diversity when W+ coefficients that are not identity-specific are manipulated. This test should be added before the mechanism-based explanation of the advantage is accepted.","section":"Sec. 2.2.2, Fig. 3"}],"minor_comments":[{"comment":"The figure labels do not match the bibliography: Fig. 3 cites \"W+ adapter [12]\" and \"IP-Adapter [32]\", but the corresponding references in the bibliography are [13] and [37]. All in-text citation numbers should be checked for consistency.","section":"Fig. 3 and references"},{"comment":"The dataset description says each identity is represented by ten images, but the training-configuration analysis in Sec. 3.4.2 uses four, six, eight, or ten training images. It should be clarified how the remaining images are used, whether the same six images are selected for all identities, and whether the test prompts overlap with any training images.","section":"Sec. 2.3 and Sec. 3.4.2"},{"comment":"References [20] and [21] appear to be the same paper (StylRes), and references [24] and [25] are the same CLIP paper with inconsistent venue capitalization (\"PmLR\" in [25]). These should be merged or corrected.","section":"References"},{"comment":"The paper uses both `𝑤+` and `w+` for the W+ latent space, sometimes within the same section (e.g., Sec. 2.2.2 and Fig. 1). The notation should be standardized throughout.","section":"Notation"},{"comment":"The explanation that four training images cause the model to \"overfit images and text prompts\" while six images avoid overfitting is not self-evident and is not supported by the qualitative figure alone. Please clarify the expected behavior and, if possible, show quantitative evidence for the stated overfitting effect.","section":"Sec. 3.4.2"},{"comment":"The paper refers to providing the dataset and code in the supplementary material but does not state whether they will be publicly released or under what license. This information should be explicit for reproducibility.","section":"Sec. 2.3 and Sec. 3.3.1"}],"recommendation":"major_revision","confidential_remarks":"The architectural proposal is plausible and the paper has a clear motivation, but the evaluation protocol is the main obstacle. The per-method reference-scoring rule and the selection of λ and the training-image count on the evaluation set are load-bearing issues that require a rerun of the comparison under a uniform, held-out protocol. I do not see a fundamental flaw in the method itself from the manuscript; the claims are simply stronger than the evidence currently supports. The authors should be asked to provide fixed-reference scoring for all baselines, multiple-seed statistics, and a disjoint evaluation split before this can be considered for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new piece here is the Style Cross-Attention block placed after the text cross-attention block, taking the text-conditioned query and injecting the W+ visual prompt. That ordering is a clean, testable idea, and the ablation against the parallel cross-attention design (Table 2) gives independent evidence that the sequential placement helps. The paper is also clearly written, and the attention-map visualizations in Fig. 3 do a decent job of illustrating the localization failure they are trying to fix.\n\nThat said, the quantitative comparison does not hold up as reported. The stress-test note is on target. Lambda = 0.4 and the six-image training set are chosen in Sec. 3.4.2 and 3.4.4 using the same ID/CLIP-T scores that later appear in Table 1, with no held-out split described. That is textbook tuning-on-the-test-set, and it makes the headline numbers very hard to interpret. On top of that, for Textual Inversion, Custom Diffusion, and DreamBooth the ID score is computed against the most similar training face, which is a per-sample max over a small gallery. That is a different metric from the fixed-reference protocol used for IP-Adapter, W+ Adapter, and PIDiff, so the rows are not comparable. And even between the two most similar methods, the margins are razor-thin: CLIP-T 0.1938 vs 0.1935, LPIPS 0.5936 vs 0.5947. Without error bars or multiple seeds, those differences are plausibly run-to-run noise.\n\nThe e4e-disentanglement assumption is a real risk, but not the main problem. Even if W+ were perfectly identity-only, the comparative claim would still be unsupported because the numbers are produced by a tuned, method-dependent evaluation. The dataset, collected from Google without consent details, is also a minor concern; if the paper is to be built upon, that needs to be sorted out.\n\nWho gets value from this? People working on W+-space adapters or attention-placed conditioning for diffusion will find the SCA ordering a useful point of comparison. The paper deserves a serious referee because the core architectural idea is testable and the writing is honest, but the experimental section needs a real overhaul before the claims can be trusted. I would not cite it as a reliable benchmark until the evaluation, code, and dataset are made public with a fixed reference protocol and held-out hyperparameter selection.","headline":"PIDiff adds a sensible sequential cross-attention block to W+-based identity customization, but its quantitative claims are undermined by evaluation choices that favor the method.","tokens_in":13794,"tokens_out":1515,"would_cite":false,"duration_ms":17280,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PIDiff claims that feeding StyleGAN's W+ latent vectors through a dedicated cross-attention block preserves a person's identity in text-to-image generation better than text embeddings or CLIP image patches, while keeping the pretrained…","keywords":["personalized text-to-image generation","identity preservation","W+ latent space","diffusion model fine-tuning","cross-attention","style editing","StyleGAN inversion","face generation"],"falsifier":"Take faces photographed against many different backgrounds, invert each cropped and aligned face to W+, and decode the W+ code with a StyleGAN generator: if the decoded image reconstructs background or scene structure rather than only the face, then scene information does leak into the visual prompt. Equivalently, measure the overlap, such as IoU, between PIDiff's visual-prompt attention map and the face segmentation mask across prompts; if that overlap drops sharply on cluttered backgrounds, the localization claim would be contradicted.","tokens_in":12726,"feed_emoji":"🎨","tokens_out":8504,"duration_ms":75056,"temperature":0.7,"pith_summary":"This paper proposes PIDiff, a fine-tuned diffusion model that represents a person's identity with a StyleGAN W+ latent vector rather than with a text embedding or a CLIP image patch. The paper argues that prior visual-prompt methods let identity and background entangle, so generated faces lose detail and the reference scene leaks into the output; PIDiff instead maps W+ codes through a Visual Guidance Module and injects them with a Style Cross-Attention block placed after the text cross-attention block. Only the new attention projection matrices and the mapping network are trained, for a few hundred steps, so the pretrained Stable Diffusion keeps its ability to follow text. On the authors' balanced 27-identity dataset, PIDiff reports the highest identity similarity among the compared methods (ID 0.3112) while matching the best text-image alignment (CLIP-T 0.1938), and it can edit style by concatenating coarse and fine W+ segments. The intended consequence is that personalized identity generation can change background, pose, clothing, and style according to the prompt without losing the person's face.","feed_headline":"W+ face codes beat image patches for identity generation","feed_subtitle":"PIDiff routes StyleGAN's W+ vectors through new cross-attention, holding the face while the scene follows the text.","key_machinery":"The central mechanism is the Style Cross-Attention (SCA) block inserted after each text cross-attention block. It takes the text-conditioned hidden state as its query and the W+-derived visual embeddings as its keys and values, so the visual prompt binds to the region the text attention has already distinguished rather than spreading over the whole background. The Visual Guidance Module supplies those visual embeddings by splitting the 18x512 W+ code into four groups of latent codes, mapping each group through a separate layer into a 768-dimensional token, and concatenating the results, which preserves the coarse-to-fine structure of the W+ space. Only the SCA projection matrices and the mapping network are trainable, and a scalar lambda controls visual-prompt strength during inference, with the paper choosing lambda=0.4 as the balance point.","core_discovery":"The central claim is that identity information can be cleanly separated from background and scene information by representing a face as a W+ latent vector from StyleGAN rather than as text or CLIP image-patch embeddings. PIDiff crops and aligns the input face, inverts it with the e4e encoder to an 18x512 W+ code, and maps that code through a Visual Guidance Module into four 768-dimensional visual tokens. The tokens enter the diffusion U-Net through a Style Cross-Attention block placed after each text cross-attention block, with the text-conditioned hidden state serving as the query; this ordering is what confines the visual prompt to the face region. During training only the new SCA projection matrices and the mapping network are updated, with random W+ noise and a 0.5 drop probability, so the pretrained Stable Diffusion keeps its text-to-image abilities. On the paper's 27-identity dataset, PIDiff reports identity similarity 0.3112, LPIPS 0.5936, and CLIP-T 0.1938; W+ Adapter, the closest rival, reports 0.2668, 0.6774, and 0.1935, and the other compared baselines score lower on identity.","pith_inferences":["Inference: The paper's advantage would be more convincing if W+ disentanglement were measured directly; decoding the inverted W+ code and checking whether background structure is recoverable would settle whether identity and scene information are really separated.","Inference: SCA's query-from-text-cross-attention design is a general recipe: any condition that should bind to a region, not just a face, could be injected the same way, potentially extending PIDiff to objects, logos, or scene elements.","Inference: The global lambda scalar could be made layer-specific or timestep-dependent; the reported lambda=0.4 result suggests a schedule that raises identity weight at early denoising steps and lowers it later might improve both metrics beyond the single trade-off point.","Inference: The 27-identity dataset is small and self-constructed; numbers like ID 0.3112 are meaningful relative to the baselines in the table, but the general ranking would be on firmer ground if reproduced on an independent multi-image-per-identity benchmark."],"forward_implications":["Using W+ prompts instead of text or CLIP image-patch prompts raises identity similarity from 0.2858 (IP-Adapter) and 0.2668 (W+ Adapter) to 0.3112 on the paper's dataset, with CLIP-T essentially tied at 0.1938.","Because only the SCA projection matrices and the mapping network are trainable, PIDiff preserves the pretrained model's text-following ability, so per-identity tuning stays light and does not destroy the base model.","W+ vectors are organized coarse-to-fine, so combining layers 1-9 of one identity with layers 10-18 of another edits visible style while keeping identity, and adding a style image to the training set gives a similar effect.","Six training images gave the best results; four overfit and eight or ten make identity harder to learn.","A lambda of 0.4 in SCA balances identity and text alignment; smaller lambda loses identity, while larger lambda drops CLIP-T sharply."],"supporting_citations":[{"why":"Provides the Stable Diffusion latent diffusion backbone that PIDiff fine-tunes for identity generation.","marker":"[27]"},{"why":"Supplies the e4e encoder that maps an aligned face to the W+ latent code used as PIDiff's visual prompt.","marker":"[32]"},{"why":"The W+ Adapter baseline that also combines W+ space with diffusion; PIDiff's reported gains are measured against it.","marker":"[13]"},{"why":"IP-Adapter baseline that uses CLIP image-patch embeddings; the paper argues this entangles identity with background.","marker":"[37]"},{"why":"Textual Inversion baseline that optimizes text embeddings; it anchors the claim that text space alone loses identity.","marker":"[5]"},{"why":"Custom Diffusion baseline and the observation that cross-attention blocks dominate fine-tuning behavior, motivating SCA.","marker":"[10]"},{"why":"DreamBooth baseline using unique identifiers, a comparison point for identity preservation.","marker":"[29]"},{"why":"VICO baseline using a separate visual condition; compared in the experiments.","marker":"[6]"},{"why":"BLIP-Diffusion baseline that generates from the text embedding space; compared in the experiments.","marker":"[11]"},{"why":"ArcFace face-recognition network used to compute the identity-similarity metric that ranks the methods.","marker":"[4]"}],"fun_headline_variants":["W+ latents keep identity, not background, in diffusion","W+ vectors, not image patches, for identity-preserving diffusion","PIDiff: StyleGAN W+ space disentangles identity from scene","Identity via W+ codes: PIDiff beats image-patch baselines","Diffusion gets clean identity via StyleGAN's W+ space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole improvement depends on the claim that the e4e encoder's W+ code for a cropped, aligned face captures identity without also carrying background or scene content; the paper supports this with attention-map pictures, not with a quantitative test of disentanglement.","fun_headline_variants_meta":{"raw":{"variants":["W+ latents keep identity, not background, in diffusion","W+ vectors, not image patches, for identity-preserving diffusion","PIDiff: StyleGAN W+ space disentangles identity from scene","Identity via W+ codes: PIDiff beats image-patch baselines","Diffusion gets clean identity via StyleGAN's W+ space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001441,"raw_usage":{"total_tokens":5869,"prompt_tokens":1068,"completion_tokens":4801,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":684,"completion_tokens_details":{"reasoning_tokens":4711}},"tokens_in":684,"tokens_out":4801,"duration_ms":32116,"temperature":1.0,"reasoning_tokens":4711,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:13:46.606412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take faces photographed against many different backgrounds, invert each cropped and aligned face to W+, and decode the W+ code with a StyleGAN generator: if the decoded image reconstructs background or scene structure rather than only the face, then scene information does leak into the visual prompt. Equivalently, measure the overlap, such as IoU, between PIDiff's visual-prompt attention map and the face segmentation mask across prompts; if that overlap drops sharply on cluttered backgrounds, the localization claim would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The W+ Adapter baseline that also combines W+ space with diffusion; PIDiff's reported gains are measured against it."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BLIP-Diffusion baseline that generates from the text embedding space; compared in the experiments."}],"review_version":1}