{"id":"612480e3-9b8e-4715-b7cf-96eedc368267","arxiv_id":"2411.19156","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A hypernetwork generates a per-instruction LoRA from a before-after image pair, and a reverse training loss allows learning from paired data alone.","lead":"This paper introduces LoRA of Change, a system that learns to generate a small adapter (LoRA) from a before-and-after image pair, then applies it to edit new query images with the same visual change. It is worth reading because it shows paired images alone, without expensive four-image training data, may be enough to train general visual instruction editors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (7) reverse-loss assumption unvalidated: training queries are horizontal flips of A, so the loss may be satisfied by appearance shortcuts and not by encoding the semantic edit.","rationale":"The reader identified the same load-bearing assumption; I partially agree, and I sharpen it with the horizontal-flip construction that makes the inverse assumption especially easy to satisfy by appearance shortcuts. The proposed test directly measures whether the inverse property transfers outside the training distribution. This does not impugn the method's qualitative results; it only says the central generalization claim needs an explicit validation. I therefore keep the conditional verdict unchanged.","tokens_in":12145,"tokens_out":7966,"duration_ms":73547,"concrete_test":"Using the released/retrained model and 100 held-out before-after pairs from the InstructPix2Pix test split, compute Δ=H(A,A'), edit B to B'_pred=G(Δ,B) with DDIM, and then cycle back with B_cycle=G(-Δ,B'_pred). Measure cycle error (LPIPS or MSE) for (i) B=flip(A) and (ii) the actual arbitrary query B in the test set. If the arbitrary-query cycle error is much larger than the flip-query error, Eq. (7) only works on the training distribution and the inverse-LoRA assumption is not validated; if the two errors are comparable, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a single before-after pair (A,A') can be distilled into a reusable LoRA that edits arbitrary query images. The load-bearing step is Eq. (7), which assumes that negating the generated LoRA inverts the edit: G(-H(A,A'), B') ≈ B. This is not a formal consequence of LoRA (LoRA is a low-rank perturbation added to attention projections) nor of the DDIM sampling trajectory, and no derivation is supplied. The fragility is amplified by the paired-data construction: during training B and B' are always horizontal flips of A and A', so both loss terms are evaluated only on images mirror-symmetric to the instruction pair. The forward term can then be satisfied by a LoRA that copies the mirrored appearance of A' into B', while the reverse term can be satisfied by a complementary appearance shortcut; the loss does not force the LoRA to encode the semantic change for an arbitrary query. Random exchange of (A,A') enforces H(A,A')≈-H(A',A) on flip-augmented pairs but does not test generalization to a different query distribution. The paper's ablations (Figs. 6 and 9) are qualitative and do not quantify how much of the improvement is true disentanglement versus cycle-regularized output appearance. Thus the strongest claim, that LoC supports broad real-world visual instructions on arbitrary queries, rests on an unvalidated inverse operator.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LoRA of Change (LoC), a framework for image editing from visual instructions, where a before-after image pair (A, A') is the instruction. A hypernetwork H encodes the pair into a LoRA that is injected into a frozen InstructPix2Pix model G, which then edits an arbitrary query image B. To avoid the need for quad data (A, A', B, B'), training uses only paired data by setting B and B' to random horizontal flips of A and A'. A 'LoRA Reverse' objective is added: G(-H(A,A'), B') is trained to reconstruct B, under the assumption that negating the generated LoRA inverts the encoded edit. The authors report qualitative results across six edit types and quantitative results (LPIPS, Visual CLIP, FID, inference time) on a 1000-image subset of the InstructPix2Pix dataset, plus a small user study. They claim that the method supports a broad spectrum of real-world visual instructions and enables real-time editing without test-time fine-tuning.","tokens_in":12433,"tokens_out":4051,"duration_ms":36876,"significance":"If the central mechanism holds, the paper makes a useful contribution: a hypernetwork that distills a reusable, interpretable edit operator from a single before-after pair, trained without quad data. This could broaden the applicability of visual-instruction editing and partially address the quad-data scarcity that limits prior inpainting- and ControlNet-based approaches. The framework is clearly described and the qualitative examples for addition, manipulation, removal, style transfer, replacement, and face manipulation are promising. The paper also honestly identifies the appearance-leakage problem and proposes a regularization strategy. However, the validity of the core 'LoRA Reverse' assumption is not established, and the quantitative evidence is thin. The paper does not release code, which limits reproducibility, but that is not itself a reason for rejection. Overall, the idea is interesting and likely within the scope of the journal if the load-bearing assumptions and evaluation gaps are addressed.","major_comments":[{"comment":"The reverse objective assumes that negating the generated LoRA exactly inverts the edit, i.e., G(-H(A,A'), B') reconstructs B. This is not a formal consequence of LoRA (a low-rank additive weight perturbation) or of the DDIM generation process, and no derivation or direct validation is provided. Furthermore, because B and B' are always horizontal flips of A and A' during training, both loss terms are evaluated only on queries that are mirror-symmetric to the instruction pair. A model could satisfy the forward and reverse losses by appearance shortcuts (e.g., copying the mirrored appearance of A' into B' and vice versa) without encoding the semantic change in a way that transfers to arbitrary query images. Please provide concrete evidence that the learned LoRA actually inverts the edit on arbitrary queries, for example by evaluating on true quad data with B not equal to a flip of A, or by probing the effect of the generated LoRA and its negation on held-out queries. Without this, the central claim of generalizable instruction extraction from paired data alone is not established.","section":"Section 3.2, Eq. (7)"},{"comment":"The quantitative evaluation is based on a single 1000-image subset with no error bars, confidence intervals, or significance tests. The claim of 'consistently better LPIPS, Visual CLIP, and FID' is not supported by variance estimates; notably, VISII achieves a higher Visual CLIP score, and the authors' explanation (test-time fine-tuning) is plausible but not tested. Moreover, the Visual CLIP metric — cos[(M(B')-M(B)), (M(A')-M(A))] — could in principle be satisfied by an output that copies the appearance of A' rather than by learning the edit direction; please discuss this risk and report additional metrics that separate edit fidelity from appearance leakage. The six claimed edit types also need per-category quantitative results, not only qualitative figures.","section":"Section 4.3, Table 2"},{"comment":"The reusability and disentanglement claims are supported only by qualitative examples. The random-exchange consistency regularization implicitly encourages H(A,A') = -H(A',A), but this is demonstrated only for flip-augmented pairs and does not establish that the extracted LoRA is free of appearance information for arbitrary queries. Please add a quantitative disentanglement evaluation, such as measuring the perceptual distance between the edited output and the query in regions that should remain unchanged, and the edit-direction alignment separately on unseen queries. Without such metrics, the claim that 'the learned instruction-specific LoRA only encodes desired visual instruction' remains unsupported.","section":"Section 4.4, Figures 8 and 9"},{"comment":"The LoRA rank r and the LoRA scale factor are never specified, and the total number of trainable parameters in the hypernetwork is not reported. These hyperparameters directly control the capacity of the generated LoRA and thereby affect the plausibility of the reverse-inversion assumption. Please provide these values, along with the random-exchange probability used during training, so that the experimental setup is reproducible and the scope of the claimed 'parameter-free' or hyperparameter-insensitive behavior is clear.","section":"Section 3.1 and Appendix A"}],"minor_comments":[{"comment":"Please report the wall-clock training time and the total model size, in addition to the number of GPUs and epochs.","section":"Section 4.1"},{"comment":"The notation Θ(x_t, t, B, H(A,A')) is slightly ambiguous: clarify that H(A,A') is injected as LoRA weights into the UNet, while B is the spatial condition.","section":"Section 3.2, Eq. (6)-(7)"},{"comment":"The user study reports only aggregate preference percentages (87.4% for LoC). Please report the number of votes per condition, inter-rater agreement, and the distribution across the six edit types, as well as whether the preference is statistically significant.","section":"Section 4.3, user study"},{"comment":"The ablation for stage-2 fine-tuning reports only the Visual CLIP score (0.193 to 0.214). It would be more informative to report LPIPS and FID as well.","section":"Section 4.4, two-stage training"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the journal and the core idea is interesting, but the load-bearing assumption behind the LoRA Reverse objective (negation inverts the edit) is unvalidated, and the quantitative evidence is thin. The paper's only self-citation (DAC, ref [25]) is used for test images and does not appear to bias the central claims. I would encourage the editor to invite a revision that addresses the reverse-loss validation on true quad data, adds error bars and per-category numbers, and reports key hyperparameters such as the LoRA rank and scale. The qualitative results are strong enough to merit a resubmission after these points are handled."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear X,\n\nThe one thing to know: LoC is a genuine new mechanism, not a repackaging. A hypernetwork reads a before-after pair and emits LoRA weights for a frozen InstructPix2Pix; training uses only paired images by adding a reverse loss that negates the generated LoRA and asks the model to reconstruct the before image. That is a clever way around the quad-data bottleneck, and the qualitative results across addition, removal, style transfer, face manipulation etc. are the best I've seen from a single model on this task. The interpretability/reusability demo — one instruction pair applied to multiple queries — is convincing.\n\nHowever, the reverse loss is the load-bearing wall. Eq. (7) assumes -H(A,A') behaves as the exact inverse of the edit, and LoRA weights give you no reason to expect that. Negating a low-rank perturbation is not the same as inverting a diffusion trajectory. The paper never derives or validates this. Worse, training sets B and B' as horizontal flips of A and A', so both loss terms are evaluated only on images mirror-symmetric to the instruction pair. The model could satisfy both terms by copying/mirroring appearance rather than encoding the semantic change. The random-exchange trick enforces H(A,A')≈-H(A',A) on that same limited distribution, but doesn't test generalization to arbitrary queries. So the central claim — a single pair yields a reusable operator for arbitrary images — rests on an unvalidated assumption.\n\nThe quantitative evidence is also thinner than the prose. One 1000-image subset, no error bars or significance tests, a 30-sample/43-worker user study with only aggregate preference rates, and no per-category breakdown for the six edit types. The ablations are qualitative and don't quantify how much of the gain is true disentanglement versus cycle-regularized output appearance. No code or weights, so nothing is independently checkable.\n\nNone of this is fatal. The idea is original enough and the empirical results are promising enough that the paper deserves serious refereeing. It should be sent to review with clear asks: show the reverse-loss assumption holds (or approximately holds) on held-out query distributions, evaluate with confidence intervals per edit type, and release the artifacts.","headline":"New paired-data-only visual-instruction editing method with a plausible but under-validated inverse-LoRA training trick; worth refereeing, not yet convincing.","tokens_in":12952,"tokens_out":2254,"would_cite":true,"duration_ms":19693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"From one before-after image pair, the LoRA of Change framework dynamically generates a low-rank edit operator that applies to any query image, trained on paired data alone.","keywords":["image editing","visual instructions","low-rank adaptation","hypernetwork","diffusion models","paired-data training","appearance leakage","instruction transfer"],"falsifier":"Take a trained model and a held-out before-after pair, generate $B' = G(H(A,A'), B)$, then apply the negated adapter to get $G(-H(A,A'), B')$, and measure how close the result is to $B$. If a substantial fraction of pairs fails this round-trip, the central assumption behind the LoRA Reverse objective is violated.","tokens_in":11951,"feed_emoji":"🖼️","tokens_out":8371,"duration_ms":75465,"temperature":0.7,"pith_summary":"Image editing usually relies on language prompts, but a before-after image pair can carry an edit instruction more precisely than words. This paper argues that the 'change' in such a pair can be compressed into a LoRA—a small set of low-rank weights added to a frozen diffusion model—by a hypernetwork that reads the two images and outputs the adapter. The paper's central move is a LoRA Reverse training objective: the same adapter, negated, must turn the edited image back into the original, which stops the model from merely copying the after image's appearance. With that regularizer, the model can be trained on paired data alone instead of scarce quadruples, and at inference the generated adapter edits any query image without fine-tuning.","feed_headline":"One before-after pair yields a reusable image-edit LoRA","feed_subtitle":"Trained on paired data alone, the model edits unseen query images in real time without fine-tuning.","key_machinery":"The central object is the instruction-specific LoRA (low-rank weight adapter) generated by the hypernetwork $H$. $H$ reads the pair $(A,A')$ with a vision-transformer encoder, fuses the features, and uses a transformer decoder with learnable queries plus layer-wise linear projections to emit the two low-rank matrices for each attention layer of the frozen InstructPix2Pix model. The reverse training objective is a second diffusion reconstruction loss evaluated with the negated LoRA, which suppresses appearance leakage, and randomly swapping $A$ and $A'$ encourages $H(A,A') = -H(A',A)$.","core_discovery":"The paper's central claim is that a visual editing instruction can be isolated as a dynamic LoRA: given a before-after pair $(A, A')$, a hypernetwork $H$ outputs adapter weights so that the frozen diffusion model satisfies $B' = G(H(A,A'), B)$ for any query $B$. The LoRA Reverse objective adds the constraint that the negated adapter must invert the edit, so that $G(-H(A,A'), B')$ reconstructs $B$, and random exchange of the pair during training encourages anti-symmetry. The paper argues this disentangles the change from the appearance of the specific images, so the same adapter transfers across query images and training can be done with only paired data, covering a broad spectrum of edit types.","pith_inferences":["Editorial extension: because the edit lives in additive LoRA weights, adapters from two different instructions could be added to perform both edits at once; the paper does not test composition.","Editorial extension: the reverse objective could be turned into a direct round-trip evaluation on held-out pairs, measuring how close $G(-H(A,A'), G(H(A,A'), B))$ is to $B$; this would stress-test the central assumption.","Editorial extension: if the hypernetwork's feature space is shared across instructions, the same architecture may handle edit types beyond the six reported, but that is an empirical question the paper leaves open."],"forward_implications":["Training can scale to a broad class of real-world visual instructions because each example needs only a before-after pair, not a before-after-query-target quadruple.","At inference the model needs no test-time fine-tuning: one forward pass through the hypernetwork produces an adapter that edits an arbitrary query image.","The same generated adapter is reusable across different query images, so an edit instruction is captured once and applied many times.","The reverse-loss regularizer plus random exchange prevents the edited output from simply copying the after image's appearance, which is what makes paired-data training feasible."],"supporting_citations":[{"why":"Frozen InstructPix2Pix diffusion model into which the generated LoRAs are injected; it is the generator G used in both forward and reverse losses.","marker":"[3]"},{"why":"Defines LoRA low-rank adaptation and the zero-initialization scheme adopted for the layer-wise projections to ensure the generated adapter starts as zero.","marker":"[12]"},{"why":"VISII baseline that inverts visual instructions into text embeddings; the approach the paper contrasts with its dynamic LoRA extraction.","marker":"[19]"},{"why":"Analogist baseline for visual-instruction editing used in qualitative and quantitative comparisons.","marker":"[9]"},{"why":"SEED-Data-Edit hybrid paired dataset used for large-scale stage-1 pretraining.","marker":"[7]"},{"why":"MagicBrush curated paired dataset used for stage-2 finetuning.","marker":"[32]"},{"why":"DDIM sampling procedure used at inference to generate the edited image from the adapter and query.","marker":"[24]"}],"fun_headline_variants":["One before-after pair, any photo: instant edit via LoRA","Learn an edit from one pair, apply it to all images","A single before-after pair yields a reusable edit LoRA","Dynamic LoRA from one pair: edit any query in real time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that negating the generated low-rank adapter exactly reverses the edit, so $G(-H(A,A'), B')$ reconstructs $B$; if that is false, the reverse loss can be satisfied without the adapter truly isolating the change.","fun_headline_variants_meta":{"raw":{"variants":["One before-after pair, any photo: instant edit via LoRA","Learn an edit from one pair, apply it to all images","A single before-after pair yields a reusable edit LoRA","Dynamic LoRA from one pair: edit any query in real time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001136,"raw_usage":{"total_tokens":4687,"prompt_tokens":882,"completion_tokens":3805,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":3733}},"tokens_in":498,"tokens_out":3805,"duration_ms":24627,"temperature":1.0,"reasoning_tokens":3733,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:28:03.908360+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained model and a held-out before-after pair, generate $B' = G(H(A,A'), B)$, then apply the negated adapter to get $G(-H(A,A'), B')$, and measure how close the result is to $B$. If a substantial fraction of pairs fails this round-trip, the central assumption behind the LoRA Reverse objective is violated.","supporting_citations":[{"cited_title":"In- structpix2pix: Learning to follow image editing instructions","cited_arxiv_id":null,"evidence_quote":"Frozen InstructPix2Pix diffusion model into which the generated LoRAs are injected; it is the generator G used in both forward and reverse losses."},{"cited_title":"Visual instruction inversion: Image editing via visual prompting","cited_arxiv_id":null,"evidence_quote":"VISII baseline that inverts visual instructions into text embeddings; the approach the paper contrasts with its dynamic LoRA extraction."},{"cited_title":"Analogist: Out-of-the-box visual in-context learning with image diffusion model","cited_arxiv_id":null,"evidence_quote":"Analogist baseline for visual-instruction editing used in qualitative and quantitative comparisons."},{"cited_title":"Magicbrush: A manually annotated dataset for instruction- guided image editing","cited_arxiv_id":null,"evidence_quote":"MagicBrush curated paired dataset used for stage-2 finetuning."}],"review_version":1}