{"id":"bb848ed0-6bee-4303-bb5d-038c8fc33c4f","arxiv_id":"2412.05619","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ONE-PIC unifies diffusion fine-tuning tasks by arranging source and target images into a single visual context and training SDXL with LoRA to predict only the masked target region.","lead":"ONE-PIC fine-tunes a pretrained image diffusion model for many different tasks by stitching the input and output images into one large image and teaching the model to fill in a masked target region. The authors claim this needs no extra network branches, uses only about 0.6% new parameters, and matches task-specific systems at a fraction of the training cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'comparable performance' claim is supported only by curated qualitative figures and an unspecified resource ratio; no quantitative metrics or matched baselines are reported, so the central claim is not empirically established.","rationale":"The reader's weakest_assumption focuses on the method's internal mechanism—whether self-attention in a fine-tuned SDXL U-Net can extract the required visual information from a concatenated grid. My concern is adjacent but distinct: even if the mechanism is plausible, the paper provides no quantitative evidence that it achieves the claimed performance parity. This is the most load-bearing issue because every headline claim ('comparable results', '2% resources', 'faster convergence') is either qualitative or insufficiently specified. The conditional verdict is appropriate: the framework is simple and potentially valuable, but acceptance should require quantitative evaluation and matched baselines. A concrete re-implementation on VITON-HD with standard metrics and a user study would settle whether the central claim actually holds. Until then, the evidence for parity is absent, not merely weak.","tokens_in":11478,"tokens_out":3570,"duration_ms":37079,"concrete_test":"Reproduce ONE-PIC on the VITON-HD test set at the same resolution as StableVTON and report LPIPS, SSIM, FID, and a user preference study against the released StableVTON baseline. Predefine a non-inferiority margin (e.g., FID within 5% of baseline). If ONE-PIC fails the margin, the 'comparable performance' claim is refuted; if it passes, the concern is resolved. The same protocol should be applied to DreamBench for DreamBooth and to a pose/depth benchmark for ControlNet to establish generality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that ONE-PIC performs comparably to task-specific fine-tuning methods with only 0.618% learnable parameters and a fraction of training cost—rests on qualitative examples in Figures 6, 8, 9, and 10, and on a convergence comparison (Figure 7) that reuses ControlNet's published curves without measuring output quality. No quantitative evaluation (LPIPS, SSIM, FID, CLIP score, DINO, or user studies) is provided for any of the four main tasks in Sections 4.1–4.4. The '2% of StableVTON resources' claim appears in the abstract and introduction without specifying what resource is measured (GPU-hours, memory, steps) or how it was computed. Because the method's novelty is precisely that a single masked-inpainting objective with LoRA-tuned self-attention can replace task-specific architectures, the absence of measured performance against standard baselines leaves the strongest claim unfalsifiable as presented. The core assumption that inpainting with stitched visual context is sufficient for fine-grained spatial control and semantics cannot be checked from the current evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ONE-PIC, a unified fine-tuning framework for large pretrained diffusion models. The method keeps the SDXL architecture unchanged, constructs a task-specific visual context by stitching reference and target images into a single grid, applies a masking strategy that adds noise only to the target region, and fine-tunes only LoRA adapters on the self-attention layers. The training objective is the standard masked diffusion noise-prediction loss of Eq. (3). The authors claim that this single recipe achieves performance comparable to task-specific methods across visual conditional control, DreamBooth, image editing, virtual try-on, and several additional tasks, while using only 0.618% learnable parameters and, for virtual try-on, about 2% of the resources required by StableVTON. The experimental sections present qualitative results (Figures 6, 8, 9, 10, 11), a convergence comparison with ControlNet (Figure 7), and per-task training times and data subsets.","tokens_in":11749,"tokens_out":3184,"duration_ms":38297,"significance":"If the central claims were quantitatively established, the contribution would be significant: a single, parameter-efficient fine-tuning recipe that leverages pretrained diffusion knowledge across multiple image-generation tasks would simplify adaptation, reduce training cost, and lower the barrier for downstream users. The method is simple and clearly described, the loss in Eq. (3) is a standard and sound masked diffusion objective, the architecture introduces no invented modules, and the code is publicly released. However, the significance is currently conditional: the claims of 'comparable performance' and '2% resources' rest on curated qualitative figures and unspecified resource accounting, with no quantitative evaluation against baselines. The paper also relies on task-specific visual-context layouts, which partially qualifies the 'one framework for all tasks' narrative. The potential is real, but the evidence presented is not yet sufficient for a journal-level claim.","major_comments":[{"comment":"The central claim of performance comparable to task-specific fine-tuning methods is not backed by any quantitative metric. Neither FID, LPIPS, SSIM, CLIP score, DINO similarity, nor user studies are reported for any of the four main tasks. For DreamBooth, prior work evaluates on DreamBench with CLIP and DINO metrics, but Sec. 4.2 reports only qualitative examples; for virtual try-on, Sec. 4.4 reports no LPIPS/SSIM/FID against VITON-HD or StableVTON baselines; for image editing, Sec. 4.3 reports no comparison on MagicBrush. Without matched quantitative comparisons, the strongest claim in the abstract and introduction is unfalsifiable as presented and needs to be added before the paper can be evaluated.","section":"Sec. 4.1–4.4, Figures 6, 8, 9, 10"},{"comment":"The claim that ONE-PIC uses '2% of the resources required by StableVTON' is not defined or computed. The manuscript does not specify whether the resource is GPU-hours, memory, number of parameters, training steps, or a composite measure, nor does it give the arithmetic behind the ratio. Likewise, the 0.618% learnable-parameter figure in Sec. 4 is stated without specifying the denominator or comparing with the parameter counts of ControlNet, StableVTON, or other baselines. These resource-efficiency claims are load-bearing for the paper's contribution and must be made precise and reproducible.","section":"Abstract and Sec. 1"},{"comment":"The convergence comparison with ControlNet is not a controlled experiment. The authors state that they 'utilized the convergence process images from the ControlNet paper for this comparison,' but no common evaluation metric, dataset, or hardware setting is used, and the x-axes of the two panels are not defined. The claim that ONE-PIC 'had already grasped the fine-tuning task' after 400 steps is based on qualitative inspection of generated images, not on any measured generation-quality metric. A proper comparison should report, at multiple training steps, the same quantitative metric for ONE-PIC and for a baseline trained on the same data and compute budget.","section":"Sec. 4.1, Figure 7"},{"comment":"The method requires a hand-selected visual-context layout for each task (2×2 for conditional control, 1×3 for virtual try-on, 3-position layouts for DreamBooth and editing), and Sec. 4.6 provides heuristics rather than a fixed rule. This is a task-specific design choice that partially qualifies the claim of a single unified fine-tuning framework. The paper should either provide a principled selection criterion or include an ablation quantifying how much the layout choice affects performance; without this, the generality claim is weaker than stated and the per-task layouts act as additional free parameters.","section":"Sec. 3.2, Figure 5, Sec. 4.6"}],"minor_comments":[{"comment":"The notation '˜xt' is introduced without definition, and the relationship between the noised latent in Eq. (1) and the composed latent in Eq. (2) should be stated explicitly. The text description symbol 'e' should be defined consistently with standard notation (e.g., 'c' or 'text').","section":"Eq. (1)–(2)"},{"comment":"The dataset name appears as 'LIAON-Art', which is likely a typo for 'LAION-Art'. Please correct the spelling and provide the exact dataset version or URL.","section":"Sec. 4.1"},{"comment":"The description of the 'mask-given image editing scenario' is unclear. The text says the top-left images are replaced with images containing the mask information, but it does not explain how the mask is encoded (e.g., as a binary overlay, a colored region, or a separate channel) or where the target mask is applied. Please clarify.","section":"Sec. 4.3 and Fig. 9"},{"comment":"The paper alternates between 'DreamBooth' and 'Dreambooth', and between 'StableVTON' and 'StableViton'. Please unify the capitalization and check all references for consistency.","section":"Throughout"},{"comment":"The statement that the 1×3 layout 'performed better' than the 2×2 layout for virtual try-on is not accompanied by any quantitative comparison or ablation figure. Even a small table or a supplementary figure would help the reader judge the claimed difference.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly written and the proposed training scheme is technically sound, but the empirical evaluation is far below the standard expected for the central claims. The paper currently reads as a preliminary technical report: the 'comparable performance' and '2% resources' statements are not operationalized, and the comparison with ControlNet is not a controlled experiment. I would support publication after the authors add quantitative evaluations with matched baselines for at least the four main tasks, specify the resource metric and computation, and either justify or ablate the per-task layout choices. The scope of the fix is substantial but feasible, so major_revision rather than reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper proposes ONE-PIC: to adapt SDXL to a new task, stitch the conditioning images and the target into a single grid, mask the target region, add noise only there, and fine-tune the self-attention layers with LoRA on the masked denoising loss. That's the whole method, and it's refreshingly concrete. No extra branches, no custom encoders. The authors demonstrate this recipe on visual condition control, DreamBooth, image editing, and virtual try-on, plus several extra tasks, and report 0.618% learnable parameters with very fast adaptation. The writing is clear, the code is released, and the positioning against Prompt Diffusion and OmniGen is fair: those are more complex or more expensive, while this is a simplification of both.\n\nThe soft spot is exactly what the stress-test flags: the central claim rests on curated qualitative figures and an underspecified resource ratio. There is not a single quantitative metric in the paper—no FID, LPIPS, CLIP score, SSIM, or user study—for any of the four main tasks. The 'comparable performance' language is not backed by comparison with matched baselines. The '2% of StableVTON resources' appears in the abstract and introduction, but the paper never says what resource is being measured. The convergence figure borrows ControlNet's published curves rather than running a matched training run, and 'convergence' is judged from images, not an objective curve. On top of that, the visual-context layout is chosen per task based on empirical experiments, so the 'universal' framing is qualified: you still have to pick a layout, and the paper gives heuristics but no principled rule.\n\nNone of this makes the method circular or incoherent. The loss is standard, the setup is clear, and the qualitative results look plausible. The problem is that the evidence does not yet support the strength of the claims. What is missing is straightforward: pick two of the four tasks, run standard quantitative evaluation against the task-specific baseline, report FID/LPIPS/CLIP and, for try-on, the usual SSIM/LPIPS, plus specify what '2% of resources' means. An ablation of the layout choices would also help.\n\nAs it stands, I would not yet cite the quantitative claims, but the method idea is worth noting. The paper deserves a serious referee: it is a simple, plausible approach that could save people a lot of work if it holds up. I would send it to review and ask for major revisions with the quantitative evaluation above. If the numbers come back in its favor, this becomes an easy-to-adopt baseline.","headline":"Simple, plausible, and clearly written, but the headline claims are not yet supported by quantitative evidence; worth a serious referee after major revisions.","tokens_in":12227,"tokens_out":4366,"would_cite":false,"duration_ms":35992,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ONE-PIC turns every image-generation task into a masked inpainting problem and matches task-specific models with only 0.618% learnable parameters.","keywords":["ONE-PIC","diffusion models","parameter-efficient fine-tuning","LoRA","masked inpainting","visual context","virtual try-on","image editing"],"falsifier":"Run the released ONE-PIC code on VITON-HD with the standard train/test split and compute LPIPS and FID against StableVTON; if the numbers are not within a small margin of the task-specific baseline, the 'comparable performance with 2% resources' claim is refuted. A second check: fine-tune ONE-PIC on pose-to-image control with the 2x2 context and compare to ControlNet's reported results; if it cannot match on standard metrics, the claim that one inpainting-style recipe replaces task-specific branches fails.","tokens_in":11302,"feed_emoji":"🎨","tokens_out":8313,"duration_ms":66655,"temperature":0.7,"pith_summary":"ONE-PIC claims that a single, parameter-light fine-tuning recipe can replace task-specific diffusion-model designs across very different image-generation jobs. Instead of adding branch networks, task-specific losses, or per-task training strategies, it arranges the reference or condition images and the target image into one composite visual context, masks the target region, and fine-tunes only the self-attention layers of a pretrained SDXL with LoRA to predict the noise in that masked region. On visual conditional control, DreamBooth-style subject generation, instruction-based image editing, and virtual try-on, the paper reports performance comparable to task-specific methods while using about 0.618% learnable parameters and a small fraction of the training time and data, including roughly 2% of the resources of StableVTON for virtual try-on. A sympathetic reader should care because the claim is that the bottleneck in adapting diffusion models is not new architecture but how the task is packaged as an inpainting problem.","feed_headline":"One recipe matches task-specific diffusion models at 0.618% params","feed_subtitle":"Stitching source and target into one visual context and masking the target: one tiny LoRA fine-tune covers many tasks.","key_machinery":"In-Visual-Context Tuning is the central object: it arranges source and target images into a single image so that downstream fine-tuning resembles the model's pretraining objective, letting the network read both fine-grained spatial detail and high-level semantics from the grid. The Masking Strategy is the second load-bearing piece: a masked noising process (Eq. 1-2) adds noise only to target-region latents and evaluates the DDPM loss only on that masked region (Eq. 3), turning every task into the prediction of masked parts. The third piece is LoRA applied to the U-Net self-attention layers (attn1.q/k/v, rank 32, alpha 4), which keeps learnable parameters at 0.618%. The paper also documents layout design choices, such as 2x2 grids for precise positional control, center placement for tasks with several conditions, and 1x2 or 2x1 grids for faster inference, as evidence that visual context plays a role analogous to prompts in language models.","core_discovery":"On its own terms, the paper's discovery is that large pretrained text-to-image diffusion models already contain the machinery needed for many downstream generation tasks; what was missing is a way to present those tasks in the model's native format. ONE-PIC presents each task as masked inpainting: the source and condition images are stitched together with the target image into one image, noise is added only to the latent of the target region, and the model is trained with the standard denoising loss restricted to the masked part. Because only the LoRA weights on the self-attention query, key, and value projections are updated, the pretrained knowledge stays almost intact, and the paper argues this is why adaptation is fast and needs little data. The same procedure, with different visual-context layouts per task, is shown to handle spatial control, subject-driven generation, instruction-guided editing, and virtual try-on.","pith_inferences":["If the central claim holds, the research question for a new image-generation task shifts from 'what network do we add?' to 'how do we lay out the visual context and mask?', a design choice that can be explored systematically rather than by inventing new modules.","The analogy to prompt learning suggests a stronger possibility the paper leaves implicit: some tasks may need no fine-tuning at all if their layout and mask are chosen well, extending the zero-shot inpainting results of RePaint to broader task families.","The method's dependence on the pretrained model's spatial priors, such as its center bias and the preference for certain grid shapes, implies that tasks with unusual geometry, like non-rectangular targets or multiple disjoint regions, are the natural stress test; this is an inference, not a claim in the paper.","Combining ONE-PIC's context layout with a task-specific loss or post-processing could close any remaining gap to specialized models, since the paper's evidence is 'comparable' rather than strictly better; that is an editorial suggestion."],"forward_implications":["A single base SDXL plus a small LoRA delta can be reused across tasks, since the network architecture is identical for all downstream jobs and only the input layout and mask change.","Adaptation costs fall from days and thousands of steps to hours and hundreds of steps, making fine-tuning accessible to users without large compute budgets.","New downstream tasks can be added by deciding how to stitch the images and where to place the mask, without designing task-specific losses or side networks.","Because only self-attention LoRA weights are stored per task, model sharing and deployment become simpler than keeping separate full models.","The unified masked-inpainting formulation extends to additional tasks such as colorization, fashion extraction, image segmentation, and identity-preserved portrait generation, as the paper demonstrates qualitatively."],"supporting_citations":[{"why":"Provides the SDXL base model whose U-Net, text encoder, and autoencoder ONE-PIC inherits unchanged.","marker":"[30]"},{"why":"Supplies the latent diffusion architecture and pretrained autoencoder that ONE-PIC's masked noising operates on.","marker":"[38]"},{"why":"Defines the denoising objective (Eq. 3) that ONE-PIC restricts to the masked target region.","marker":"[11]"},{"why":"Supports the premise that inpainting ability is already embedded in pretrained diffusion models, motivating the mask-based unification.","marker":"[21]"},{"why":"Serves as the visual-conditional-control baseline whose convergence (6,000 steps) is compared with ONE-PIC's 400 steps.","marker":"[53]"},{"why":"Baseline for virtual try-on; ONE-PIC claims comparable results with about 2% of StableVTON's resources.","marker":"[13]"},{"why":"Defines the DreamBooth subject-driven generation task and the DreamBench prompts used for evaluation.","marker":"[39]"},{"why":"Supplies the MagicBrush dataset for training and evaluating instruction-guided image editing.","marker":"[52]"},{"why":"Supplies the VITON-HD dataset used to train and evaluate virtual try-on.","marker":"[4]"},{"why":"Provides the LoRA parameter-efficient fine-tuning method used to update only self-attention q/k/v projections.","marker":"[22]"}],"fun_headline_variants":["ONE-PIC turns any diffusion task into masked inpainting","One diffusion model, multiple tasks, just 0.618% params","Mask and fine-tune: ONE-PIC adapts diffusion to any task","Unified diffusion fine-tuning with a single masking trick","From control to try-on: ONE-PIC masks the target"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that a pretrained SDXL U-Net, with only its self-attention LoRA weights updated, can read a stitched grid of source and target images well enough that a single masked-inpainting objective is sufficient for tasks as different as pose control and virtual try-on; if that cross-task visual-context reading fails, the whole recipe collapses.","fun_headline_variants_meta":{"raw":{"variants":["ONE-PIC turns any diffusion task into masked inpainting","One diffusion model, multiple tasks, just 0.618% params","Mask and fine-tune: ONE-PIC adapts diffusion to any task","Unified diffusion fine-tuning with a single masking trick","From control to try-on: ONE-PIC masks the target"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000331,"raw_usage":{"total_tokens":1873,"prompt_tokens":1005,"completion_tokens":868,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":779}},"tokens_in":621,"tokens_out":868,"duration_ms":8219,"temperature":1.0,"reasoning_tokens":779,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:31:54.824021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released ONE-PIC code on VITON-HD with the standard train/test split and compute LPIPS and FID against StableVTON; if the numbers are not within a small margin of the task-specific baseline, the 'comparable performance with 2% resources' claim is refuted. A second check: fine-tune ONE-PIC on pose-to-image control with the 2x2 context and compare to ControlNet's reported results; if it cannot match on standard metrics, the claim that one inpainting-style recipe replaces task-specific branches fails.","supporting_citations":[{"cited_title":"Repaint: Inpainting using denoising diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Supports the premise that inpainting ability is already embedded in pretrained diffusion models, motivating the mask-based unification."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Serves as the visual-conditional-control baseline whose convergence (6,000 steps) is compared with ONE-PIC's 400 steps."},{"cited_title":"Stableviton: Learning semantic correspon- dence with latent diffusion model for virtual try-on","cited_arxiv_id":null,"evidence_quote":"Baseline for virtual try-on; ONE-PIC claims comparable results with about 2% of StableVTON's resources."},{"cited_title":"Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation","cited_arxiv_id":null,"evidence_quote":"Defines the DreamBooth subject-driven generation task and the DreamBench prompts used for evaluation."},{"cited_title":"Magicbrush: A manually annotated dataset for instruction- guided image editing","cited_arxiv_id":null,"evidence_quote":"Supplies the MagicBrush dataset for training and evaluating instruction-guided image editing."},{"cited_title":"Viton-hd: High-resolution virtual try-on via misalignment-aware normalization","cited_arxiv_id":null,"evidence_quote":"Supplies the VITON-HD dataset used to train and evaluate virtual try-on."},{"cited_title":"Peft: State-of-the-art parameter-efficient fine-tuning meth- ods","cited_arxiv_id":null,"evidence_quote":"Provides the LoRA parameter-efficient fine-tuning method used to update only self-attention q/k/v projections."}],"review_version":1}