{"id":"abd27b1e-9d35-47d0-bc6d-0cba2591660b","arxiv_id":"2412.11513","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"IGR restores a clean garment image from a person photo using Stable Diffusion, dual extractors, attention fusion blocks, and a VITON-to-GarmRe fine-tuning strategy, beating TryOffDiff on the reported benchmarks.","lead":"This paper introduces IGR, a diffusion-based system that takes a photo of a person in an outfit and produces a clean, standard picture of that garment alone. It improves garment restoration for virtual try-on pipelines by combining two feature extractors and a coarse-to-fine training scheme.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"StreetTryOn unpaired FID/KID cannot establish identity preservation in challenging scenarios; a paired identity metric is needed.","rationale":"Good-faith reading: the paper's architecture is plausible and the VITON-HD paired evaluation is solid. The most load-bearing weakness is the unpaired StreetTryOn evaluation, which the reader correctly identified. I agree with the reader: the domain gap between street photos and VITON-HD studio garments could dominate FID/KID, and those metrics cannot validate identity preservation. My stress-test sharpens this: the unpaired metric is not merely noisy, it is structurally unable to measure the claim about preserving the specific garment's identity. The concrete test I propose would add a paired identity metric, either through CLIP/DINO similarity between the generated garment and the reference person's garment region, or through a synthetic paired challenging set from VITON-HD. The paper's missing code/data and selective baselines support the CONDITIONAL status but are not the primary logical flaw. Credit where due: the paired VITON-HD results and the coarse-to-fine ablations (though SSIM drops slightly) indicate the method works in standard scenarios. Thus no change to the reader's verdict is needed.","tokens_in":10514,"tokens_out":12387,"duration_ms":113740,"concrete_test":"Compute a per-sample identity metric on StreetTryOn: crop the garment region from each test person via the mask and calculate CLIP (or DINOv2) embedding cosine similarity between that crop and the generated garment. Compare IGR vs TryOffDiff across the full test set (mean/std). If IGR does not clearly win, the Table 1 FID gains are likely due to domain alignment rather than identity preservation. Alternatively, create a synthetic paired challenging set from VITON-HD by adding occlusions or street-like compositing to test persons while keeping the original GT garment, and rerun SSIM/LPIPS/DISTS; vanishing gains would disprove the 'challenging scenarios' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes 'even in challenging scenarios such as complex garments or those with occlusions'. The only quantitative evidence for this is the unpaired StreetTryOn evaluation in Table 1. Because StreetTryOn has no paired ground-truth garments, FID/CLIP-FID/KID are computed between IGR's outputs for StreetTryOn persons and VITON-HD's real garments. These distribution-level metrics measure how similar the generated garment set is to the VITON-HD studio garment set, not whether each restored garment preserves the identity of the garment actually worn by the corresponding person. A model that generates generic category-appropriate VITON-HD-like garments could achieve low FID by ignoring the input identity entirely. Thus, the reported superiority on StreetTryOn (e.g., FID 31.05 vs 39.85, KID 8.75 vs 20.59) does not quantitatively support the identity-preservation claim for challenging cases. The VITON-HD paired metrics (SSIM 0.7895, LPIPS 0.2946, DISTS 0.2045 vs TryOffDiff 0.7576/0.4288/0.2680) are valid and do support the core claim for near-frontal, clean-background cases. But the generalization to complex/occluded scenarios rests on the unpaired metric, which is confounded by domain alignment. Additionally, the guidance scale (1.5) is justified partly by StreetTryOn performance, indicating test-set selection, further weakening that particular evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses garment restoration: given a person image and an agnostic mask, the goal is to produce a clean, standard garment that matches the garment worn in the image. The proposed IGR method builds on Stable Diffusion v1.5 and combines two feature extractors: IP-Adapter, which provides low-level garment features from a cropped masked region, and GarmNet, a UNet-shaped network that takes the person latent and mask as input and provides high-level garment semantics. These features are injected into the denoising UNet via additional cross-attention layers (low-level features) and modified self-attention layers (high-level features). Training is coarse-to-fine: the model is first trained on VITON-HD plus a self-collected in-shop dataset, then fine-tuned on a manually curated 20% subset. Experiments compare IGR against CFLD, OOTD, IDM, and TryOffDiff on paired VITON-HD metrics (SSIM, LPIPS, DISTS, FID, CLIP-FID, KID) and on unpaired StreetTryOn distribution metrics, with ablations of GarmNet, the fine-tuning strategy, and the guidance scale.","tokens_in":10783,"tokens_out":4695,"duration_ms":39346,"significance":"If the claims hold, the paper offers a practical and relatively simple recipe for garment restoration, with consistent quantitative gains over TryOffDiff on the paired VITON-HD benchmark (e.g., SSIM 0.7895 vs 0.7576, LPIPS 0.2946 vs 0.4288). The use of public components (Stable Diffusion, IP-Adapter), independent baselines, and ablations that isolate GarmNet and the fine-tuning data make the core engineering contribution transparent and reproducible in principle. The main weakness is that the headline generalization claim for complex or occluded scenarios rests on unpaired distribution metrics that cannot measure per-sample identity preservation, and two recent garment-restoration baselines discussed in the paper are absent from the quantitative comparison.","major_comments":[{"comment":"The unpaired StreetTryOn evaluation uses VITON-HD real garments as the reference distribution for FID, CLIP-FID, and KID. These are distribution-level metrics and do not establish that the generated garment preserves the identity of the garment actually worn by the corresponding street person; a model that emits generic VITON-HD-like garments can score well while ignoring the input identity. Since the abstract's claim about 'challenging scenarios such as complex garments or those with occlusions' is supported only by this unpaired evaluation plus qualitative examples, that generalization claim is not quantitatively supported. Please add a paired or per-sample identity metric for StreetTryOn (e.g., using manually annotated garment crops or a user study), or restrict the generalization claim to the VITON-HD setting.","section":"§4.2, Table 1"},{"comment":"TryOffAnyone and RAGDiffusion are discussed in Related Work as recent garment-restoration methods, but neither appears in Table 1. Because the paper claims to surpass existing methods and motivates IGR specifically against these approaches, the quantitative comparison is incomplete. Please add these baselines or explicitly justify their exclusion.","section":"§2.2, Table 1"},{"comment":"The ablation results do not fully support the text: adding GarmNet and HQFT lowers SSIM (0.7931 without GarmNet and 0.7984 without HQFT versus 0.7895 for the full model) while improving LPIPS, DISTS, and FID. The statement that GarmNet achieves 'higher scores in both reconstruction accuracy and authenticity' is therefore inaccurate for SSIM. Please explain this trade-off or revise the claim.","section":"§4.3, Table 2"},{"comment":"The guidance scale s=1.5 is selected based on performance on the same VITON-HD and StreetTryOn test sets that produce Table 1. Since the differences between s=1.5 and s=2.0 are small and not consistently in one direction (e.g., VITON-HD FID is better at s=2.0: 12.2856 versus 13.1425, while StreetTryOn CLIP-FID is slightly better at s=1.5: 10.0137 versus 10.3480), the choice of s affects the headline numbers. Please report the selection on a validation split or provide significance or variance information.","section":"§4.3, Table 3"}],"minor_comments":[{"comment":"In Eq. (4), 'Qf con, Qf con and Vf con' should read 'Qf con, Kf con and Vf con'.","section":"§3.2, Eq. (4)"},{"comment":"The self-collected paired dataset of in-shop models and garments is not described or released, which limits reproducibility; please provide details about its size, collection protocol, and license, or release the curated subset.","section":"§4.1"},{"comment":"CFLD, OOTD, and IDM are not designed for garment restoration and are evaluated via SAM2 segmentation of try-on outputs; this can introduce artifacts that unfairly disadvantage these baselines. Please state this limitation explicitly and, if possible, include a garment-restoration-specific baseline other than TryOffDiff.","section":"§4.2"},{"comment":"No code or trained weights are released; given the engineering nature of the contribution, providing code and weights would substantially strengthen the paper.","section":"General"},{"comment":"The phrase 'a Improved diffusion-based Garment Restoration method' contains a grammatical error; it should be 'an Improved'.","section":"§1"},{"comment":"The paper reports single-run metrics without confidence intervals; adding multiple seeds or bootstrap intervals would make the comparisons in Tables 1-3 more reliable.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with consistent gains on the paired VITON-HD benchmark, but the abstract overstates what the StreetTryOn evaluation can show. The missing pieces are addressable: a paired identity metric for StreetTryOn, inclusion of RAGDiffusion and TryOffAnyone, explanation of the SSIM ablation trade-off, and a validation-based choice of guidance scale. The scope fits a computer vision venue, though the contribution is incremental engineering rather than a new paradigm."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent engineering paper on garment restoration, with a plausible new architecture and solid paired-evaluation gains on VITON-HD. The StreetTryOn claim is not supported by the unpaired metric, and the missing baselines/code weaken confidence, but it deserves a real referee.\n\nThe architecture is clearly explained: IP-Adapter for low-level garment features plus a UNet-style GarmNet for high-level semantics, fused through GarmFus blocks that modify both cross- and self-attention in the Stable Diffusion denoiser. A coarse-to-fine training strategy on a filtered subset is a sensible addition. None of this appears in the cited TryOffDiff, TryOffAnyone, or RAGDiffusion papers, so the novelty claim is fair, even if every block is known.\n\nWhat is genuinely good: the paired evaluation on VITON-HD shows consistent gains over the baselines across SSIM, LPIPS, DISTS, FID, CLIP-FID, and KID. Ablations isolate the effect of GarmNet and the fine-tuning data, and the figures look consistent with the reported numbers. The paper also openly states the key limitation of the StreetTryOn evaluation: since StreetTryOn has no true garment images, it uses VITON-HD garments as the reference distribution for FID/CLIP-FID/KID. That honesty counts.\n\nThe soft spots are real but not fatal. First, the abstract claims strong performance on complex or occluded garments, but the only quantitative evidence for that is the unpaired StreetTryOn metric. Distribution-level FID cannot establish per-garment identity preservation; a model that outputs generic category-appropriate garments could score well. You would need a paired identity metric (same garment in a different pose, or a retrieval-style match) to support the generalization claim. Second, the guidance scale is selected partly by StreetTryOn performance, which is test-set selection for that specific claim. Third, the SSIM of the full model in Tables 2 and 3 is slightly lower than some ablations (0.7895 vs 0.7931 and 0.7984), and the paper does not discuss that inconsistency. Fourth, RAGDiffusion and TryOffAnyone are missing from the benchmark, and there is no code, data, or variance estimates, so I cannot independently check anything.\n\nWho is this for? People working on garment restoration, virtual try-on, and attention-based control of diffusion models. It deserves a serious referee: I would send it to peer review, but I would ask for code/data, a paired identity metric, at least one more recent baseline, and a sentence on the SSIM drop. With those, the contribution would be meaningfully stronger.\n\nMy own verdict: the core paired result is probably right, the generalization claim isn't proven, and the paper is honest enough about its own limitation that I trust the engineering.","headline":"Solid paired VITON-HD gains and clearly explained architecture; the unpaired StreetTryOn metric cannot carry the generalization claim.","tokens_in":11332,"tokens_out":2920,"would_cite":false,"duration_ms":26434,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model restores the exact garment from a person photo by fusing low-level and high-level garment features into the denoising process.","keywords":["garment restoration","virtual try-on inverse","latent diffusion model","feature fusion","attention mechanism","coarse-to-fine training","person image","occlusion handling"],"falsifier":"Collect a paired evaluation set of in-the-wild person photos alongside clean standard garment ground truths, and compute LPIPS and DISTS between IGR outputs and those true garments; if IGR does not beat TryOffDiff on those paired metrics, the claimed generalization advantage is falsified.","tokens_in":10277,"feed_emoji":"👗","tokens_out":6685,"duration_ms":49515,"temperature":0.7,"pith_summary":"The paper tries to establish that a diffusion-based model can restore a clean, standard garment image from a photo of a person wearing that garment, even when the garment is partially occluded or complex. The proposed IGR model builds on Stable Diffusion and adds two garment extractors: IP-Adapter for low-level features and GarmNet for high-level semantics. These are merged into the denoiser through attention-based GarmFus blocks. The authors argue that this design, together with a coarse-to-fine training strategy, outperforms prior garment-restoration and try-on-based baselines on both studio and street images. If true, the method would make garment restoration accurate enough for practical virtual try-on pipelines and e-commerce.","feed_headline":"Diffusion model restores the exact garment from a person photo","feed_subtitle":"Dual extractors and coarse-to-fine training preserve fabric identity and detail even with occlusion.","key_machinery":"The load-bearing mechanism is the Garment Fusion (GarmFus) block, an attention module that combines two feature streams in the Stable Diffusion UNet. Low-level garment features from IP-Adapter are injected via an additional cross-attention layer computed alongside the text-conditioned cross-attention; high-level semantics from GarmNet are concatenated spatially with the denoiser's intermediate features before self-attention, and only the fused half is returned. GarmNet itself is a copy of the denoiser's structure taking the person-image latent plus a resized mask as a 5-channel input, avoiding error-prone cropping. The coarse-to-fine training strategy is the second critical component: initial training on VITON-HD and a self-collected paired set, then fine-tuning on a filtered 20% subset with only front-facing, well-proportioned, properly visible garments.","core_discovery":"The central claim is that garment restoration quality is determined by how completely garment features are extracted from the person image and how faithfully they are injected into the latent diffusion denoiser. IGR therefore uses two complementary extractors: IP-Adapter captures low-level attributes, while GarmNet, which shares the denoiser's U-Net structure, encodes high-level semantics from the person image concatenated with a garment mask. In the GarmFus blocks, low-level features enter through an extra cross-attention layer, and high-level features are appended to the self-attention keys and values, with only the latter half retained. A coarse-to-fine training schedule first exploits standard VITON datasets, then fine-tunes on a manually curated subset that excludes occluded, truncated, or side-view garments. On VITON-HD and StreetTryOn tests, the paper reports consistent improvement over CFLD, OOTD, IDM, and TryOffDiff across SSIM, LPIPS, DISTS, FID, CLIP-FID, and KID.","pith_inferences":["Because the StreetTryOn evaluation is unpaired, the reported FID, CLIP-FID, and KID numbers likely mix domain shift with restoration error; a paired street-photo benchmark would give a stricter test of generalization.","The ablation attributes most detail gains to GarmNet, but a lighter high-level encoder or a second high-resolution IP-Adapter might achieve similar improvements at lower training cost.","Since IGR is built on Stable Diffusion, the same GarmFus and coarse-to-fine recipe should transfer directly to newer base models, so restoring quality should improve without re-designing the fusion blocks.","The manual curation of the fine-tuning subset, while effective, may not scale; an automatic filter that detects occluded, truncated, or side-view garments could make the training strategy practical for larger datasets."],"forward_implications":["If IGR works as claimed, garment restoration can be plugged into virtual try-on systems to produce standard garment images that faithfully match a reference person, improving downstream try-on realism.","The design shows that separating low-level and high-level garment features and fusing them at different attention layers is an effective recipe for inverse try-on, pointing the way for similar restoration tasks.","The coarse-to-fine training strategy demonstrates that naively training on VITON datasets harms garment-restoration models; curated data matters as much as architecture.","The method's success on StreetTryOn, if valid, implies reasonable generalization from studio to in-the-wild images, as long as occlusions are handled by the mask input.","Improvements in the base diffusion model would likely translate directly into better restoration, since IGR is built on Stable Diffusion."],"supporting_citations":[{"why":"Supplies the pretrained Stable Diffusion latent denoiser and generative prior that IGR fine-tunes.","marker":"[29]"},{"why":"Provides the low-level garment feature extractor whose features are fused through an additional cross-attention layer.","marker":"[40]"},{"why":"Is the tailored garment-restoration baseline that IGR is designed to outperform.","marker":"[34]"},{"why":"Provides the VITON-HD paired training and testing data used in the coarse stage and paired evaluation.","marker":"[4]"},{"why":"Provides the StreetTryOn test images used to evaluate generalization to in-the-wild photos.","marker":"[6]"},{"why":"Supplies the self-attention feature-concatenation technique used to fuse high-level garment semantics.","marker":"[13]"},{"why":"Is used to segment garments from person images when generating baseline inputs.","marker":"[28]"}],"fun_headline_variants":["Dual-extractor diffusion pulls exact garment from a photo","Coarse-to-fine diffusion restores garment identity under occlusion","Two-track diffusion model recovers original garment from person","Diffusion with twin extractors preserves fabric identity in restoration","Garment restoration via dual feature injection in diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that computing FID, CLIP-FID, and KID for unpaired StreetTryOn generations against VITON-HD ground-truth garments is a valid measure of garment restoration quality.","fun_headline_variants_meta":{"raw":{"variants":["Dual-extractor diffusion pulls exact garment from a photo","Coarse-to-fine diffusion restores garment identity under occlusion","Two-track diffusion model recovers original garment from person","Diffusion with twin extractors preserves fabric identity in restoration","Garment restoration via dual feature injection in diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2772,"prompt_tokens":908,"completion_tokens":1864,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":1795}},"tokens_in":524,"tokens_out":1864,"duration_ms":10222,"temperature":1.0,"reasoning_tokens":1795,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:50:48.175934+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a paired evaluation set of in-the-wild person photos alongside clean standard garment ground truths, and compute LPIPS and DISTS between IGR outputs and those true garments; if IGR does not beat TryOffDiff on those paired metrics, the claimed generalization advantage is falsified.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained Stable Diffusion latent denoiser and generative prior that IGR fine-tunes."},{"cited_title":"Viton-hd: High-resolution virtual try-on via misalignment-aware normalization","cited_arxiv_id":null,"evidence_quote":"Provides the VITON-HD paired training and testing data used in the coarse stage and paired evaluation."},{"cited_title":"Street tryon: Learning in-the-wild virtual try-on from unpaired person images","cited_arxiv_id":null,"evidence_quote":"Provides the StreetTryOn test images used to evaluate generalization to in-the-wild photos."},{"cited_title":"Animate anyone: Consistent and controllable image- to-video synthesis for character animation","cited_arxiv_id":null,"evidence_quote":"Supplies the self-attention feature-concatenation technique used to fuse high-level garment semantics."}],"review_version":1}