{"id":"f64cc5f1-9729-4da1-9b53-7e808596d39b","arxiv_id":"2411.16819","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Frame2Frame performs text-based image editing by generating a short video transition from the source image and selecting the best resulting frame, achieving competitive or better benchmark scores than single-image diffusion editors.","lead":"This paper turns image editing into a video problem: instead of generating a single edited image, it uses a video model to animate the original image step by step toward the target edit. The method, Frame2Frame, then picks the best video frame as the final edited image, which the authors say improves both edit accuracy and source preservation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline SOTA claim is not supported as stated: Table 1 reports ties, and the protocol in §5.1 manually selects the best of 15 seeds per method with no variance, while F2F's VLM frame-selection stage is itself guided by the target caption that the CLIP score also measures.","rationale":"I agree with the reader that the core premise is that a pretrained image-to-video model can realize the temporal transition described by the temporal caption. The most load-bearing vulnerability in the paper's support for that premise, however, is the evaluation protocol. The paper reports only manually selected best-of-15 results with no variance; on TEdBench the published numbers are ties; and F2F's frame-selection stage is an additional mechanism directly aligned with the target caption and therefore with the CLIP score. This makes the central SOTA claim difficult to verify from the published tables. A transparent per-seed comparison and a frame-selection ablation would settle whether the video path or the selector is doing the work. The PosEdit result is more favorable but similarly lacks error bars and does not use a pose-specific metric such as keypoint error; adding such a metric would strengthen the pose-editing claim. These are evidential rather than conceptual failures, so conditional acceptance remains the appropriate disposition.","tokens_in":15227,"tokens_out":5255,"duration_ms":52205,"concrete_test":"Re-run the TEdBench and PosEdit protocols with a fixed set of 15 seeds for every method, reporting mean, median, standard deviation, and best-of-N curves instead of only the manually selected best result. For F2F, additionally record the target CLIP score of the video's last frame, the VLM-selected frame, and a randomly chosen frame before any manual seed selection. If F2F's mean or median CLIP no longer exceeds FlowEdit or LEDITS++ on TEdBench, or if the last frame is not already competitive with those baselines, the central claim relies on selection bias rather than on temporal editing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 states that for each method and source image the authors \"manually select the best result from fifteen random seeds based on visual quality and edit accuracy.\" Table 1 then shows F2F tying FlowEdit on LPIPS and CLIP-I and tying LEDITS++/Pix2Pix-Zero on CLIP (0.63), so the abstract's claims of \"state-of-the-art\" and \"significant improvements\" are not demonstrated. The asymmetry is worse than a missing error bar: F2F uniquely includes a frame-selection stage (Section 3.3) whose VLM prompt asks for the image \"that best follows this edit\" using the target caption c, and the headline edit-accuracy metric, CLIP score, measures alignment with that same caption. The reported advantage can therefore reflect metric-aligned post-selection rather than the video generator's temporal evolution. The last-frame ablation (Table S5) shows selection changing CLIP from 0.61 to 0.63, but that is computed on already manually selected best seeds and has no error bars, so its magnitude is uncertain. Without per-seed statistics or released code, an independent check of whether the temporal path itself, rather than the selection stage, produces the claimed edits is impossible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Frame2Frame (F2F), a three-stage pipeline for text-based image editing in which (1) a VLM (GPT-4o) converts the target caption into a temporal editing caption, (2) a pretrained image-to-video model (CogVideoX I2V-5B) generates a video starting from the source image, and (3) a VLM selects the best frame from the video as the final edit. The method is evaluated on TEdBench and a newly proposed PosEdit dataset (derived from UTD-MHAD), with additional qualitative results on denoising, deblurring, relighting, and outpainting. The central claim is that modeling editing as a temporal trajectory along the image manifold yields edits that better satisfy complex prompts while preserving source content, compared with single-step image diffusion editors.","tokens_in":15454,"tokens_out":5192,"duration_ms":44037,"significance":"The idea of reusing large pretrained video generators for image editing is timely and creatively combines video-world priors with an editing objective. The paper is generally well written, provides detailed prompts and ablations in the appendix, includes a human study, and introduces a new pose-editing benchmark with ground-truth target images. These are real strengths. However, the quantitative support for the headline 'state-of-the-art' claim is currently weak: Table 1 shows ties with existing methods on all three metrics, and the evaluation protocol of manually selecting the best of fifteen seeds per method, without reporting variance, makes the reported advantages unreliable. The contribution is potentially valuable, but the evidence needs strengthening before publication.","major_comments":[{"comment":"The protocol states that for each method and source image, the authors 'manually select the best result from fifteen random seeds based on visual quality and edit accuracy.' This means the reported numbers are maxima over seeds, and the paper reports no variance or error bars. Table 1 shows F2F tying FlowEdit on LPIPS and CLIP-I and tying Imagic/LEDITS++ on CLIP; the abstract's claims of 'state-of-the-art' and 'significant improvements' therefore cannot be substantiated without per-seed statistics or statistical significance tests. Please report mean ± std over seeds and either remove the SOTA claim or demonstrate superiority.","section":"§5.1 (Evaluation Protocol)"},{"comment":"The frame-selection stage asks the VLM to choose the frame 'that best follows this edit fully and naturally' using the target caption c, and the main edit-accuracy metric is the CLIP score with that same caption. Table S5 shows that frame selection raises CLIP from 0.61 (last frame) to 0.63 (selected frame), so part of the reported edit accuracy can be attributed to metric-aligned post-selection rather than to the video generator's temporal pathway. To support the causal interpretation in Section 3.2, please evaluate a fixed-frame baseline (e.g., the last frame) or a selection procedure that does not have access to the target caption during selection, and report per-seed discrepancies.","section":"§3.3 and Table S5"},{"comment":"The abstract claims 'state-of-the-art results' and 'significant improvements,' but Table 1 shows F2F at 0.22 LPIPS (equal to FlowEdit), 0.89 CLIP-I (equal to FlowEdit), and 0.63 CLIP (equal to Imagic and LEDITS++). No baseline is clearly outperformed on any metric. Given the selection protocol, these ties do not support the stated claims. Please either supply a statistical comparison with error bars or revise the claims to say that F2F is competitive with, rather than superior to, existing methods.","section":"Abstract / Table 1"}],"minor_comments":[{"comment":"The ground-truth row reports CLIP score 0.61, while F2F achieves 0.64, exceeding the ground truth. This suggests the CLIP metric may not reflect true pose accuracy, and the evaluation would benefit from an additional pose-specific metric or a caveat.","section":"Table 2"},{"comment":"The claim that 'the same seed set is used across all methods' is ambiguous because different methods use different random processes; please clarify how seeds are matched across pipelines.","section":"§5.1"},{"comment":"Using the identical static prompt 'A person standing naturally with his arms relaxed at his sides' for all Pix2Pix-Zero tasks may disadvantage it on actions that imply large pose changes; a per-task description would be fairer.","section":"§5.3"},{"comment":"The description of the manifold visualization does not specify how the 2D PCA subspace was computed across the three sets (and noise samples) or how many images remained after manual filtering; please add details for reproducibility.","section":"Section 4"},{"comment":"The related work sentence 'These models, often termed ”world simulators” due to their emergent understanding of physical dynamics and temporal coherence.' is missing a verb; please correct the grammar.","section":"Section 2.2"},{"comment":"The results on denoising, deblurring, relighting, and outpainting are presented only qualitatively. A quantitative evaluation, even on a handful of examples, would strengthen the claim of broader applicability.","section":"Section 5.5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a timely idea and the appendix contains useful details, but the main evidence for the SOTA claim is currently not strong. The best-of-15-seeds protocol without variance is a serious concern and may be perceived as selective reporting. I would encourage the editor to ask for per-seed statistics and a neutral frame-selection baseline. Also, the absence of code/data release limits reproducibility of the PosEdit benchmark and the manifold visualization."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick bottom line: Frame2Frame is a genuinely new way to do image editing—treat the edit as a temporal trajectory in a generative video model and then pick the right frame. That reformulation is the first I'm aware of in the cited literature. The paper also contributes PosEdit, a paired ground-truth benchmark for pose editing that fills a real gap. On PosEdit, F2F clearly beats LEDITS++ on all metrics, and the human survey shows users prefer F2F for both edit accuracy and quality. So the core idea deserves to be seen by a serious referee.\n\nBut the paper overreaches. The abstract promises \"state-of-the-art results\" and \"significant improvements\"; Table 1 shows F2F ties FlowEdit on LPIPS and CLIP-I, ties Imagic and LEDITS++ on CLIP, and does not pull ahead on any TEdBench metric. That's not SOTA, it's parity. The evaluation protocol also manually picks the best of fifteen seeds per method and reports no variance, so we have no idea if the differences are real. And the stress-test concern is fair: the frame-selection VLM is told the target caption c, and the headline CLIP metric measures alignment with c. So F2F gets to post-select on the metric it is judged by. The ablation against last-frame selection shows a 0.61->0.63 CLIP gain, which suggests the temporal path itself does the heavy lifting, but that ablation inherits the same best-seed protocol and lacks error bars, so we can't judge the magnitude.\n\nNone of this makes the pipeline circular in the statistical sense—there are no fitted parameters, and the ablations are honestly reported. It does mean the paper's central quantitative claim on TEdBench is not established. The PosEdit results are the strongest evidence, and they are substantial. The other soft spots are the usual: no released code, and the additional vision tasks are qualitative only. Those are fixable.\n\nWho is this for? Researchers working on diffusion-based editing or on using pretrained generative video models for image tasks. It gives them a new baseline to beat and a reasonable benchmark for pose editing. If I were the editor, I would send this to review and ask for per-seed statistics, a metric-independent frame-selection check (e.g., human-labeled or evaluated on PosEdit against ground truth), and code release. I would not desk-reject it.","headline":"Real new idea for image editing via video generation, but the SOTA claim outruns the evidence: the TEdBench advantage is a tie, the protocol is best-of-15-seeds with no variance, and the frame selector post-selects on the same caption the metric uses.","tokens_in":15985,"tokens_out":3632,"would_cite":true,"duration_ms":32769,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frame2Frame shows that image editing can be recast as video generation, producing edits that preserve source content better than one-shot diffusion.","keywords":["image editing","video generation","image-to-video diffusion","temporal editing captions","frame selection","text-guided editing","pose editing","TEdBench"],"falsifier":"Run Frame2Frame on a benchmark of prompts whose edits cannot be described as plausible continuous motion, such as 'a chair sawed in half', 'a photo of an open door', or 'two bananas', and measure with CLIP and human raters whether any generated frame satisfies the target prompt while remaining close to the source. The claim predicts these still succeed because the video model can realize physical and semantic transformations; a systematic failure on such prompts, where the earliest matching frame already drifts from the source, would refute the generality of the temporal reformulation.","tokens_in":15007,"feed_emoji":"🎬","tokens_out":5933,"duration_ms":52663,"temperature":0.7,"pith_summary":"This paper argues that text-based image editing should be reformulated as a temporal process: instead of asking a diffusion model to produce one edited image in a single step, the edit is generated as a short video that starts from the source image and gradually evolves into the requested scene. The authors build Frame2Frame, a three-stage pipeline that uses a vision-language model to write a temporal editing caption, a pretrained image-to-video diffusion model to realize that transition, and a second vision-language pass to pick the earliest frame that completes the edit. On the TEdBench benchmark and on a new pose-editing benchmark called PosEdit, they report that this video-based route matches or exceeds existing single-image editors in both edit accuracy and source preservation, and the same recipe extends without retraining to deblurring, denoising, outpainting, and relighting. The reason to care is practical: if the claim holds, hard edits that usually destroy the original image can be achieved by borrowing the temporal coherence that video models already learned.","feed_headline":"Video-based image editing outperforms one-shot diffusion on fidelity","feed_subtitle":"Turning an edit into a short video transition and selecting the right frame yields edits that match the prompt and keep the source intact.","key_machinery":"The load-bearing object is the Temporal Editing Caption, a rewritten prompt that turns a static edit request into a description of motion or change over time, produced by a vision-language model with in-context examples. Together with a pretrained image-to-video diffusion model (CogVideoX I2V-5B) that treats the source image as the first video frame, this caption creates a candidate trajectory of frames on the image manifold. The third mechanism is automated frame selection: sampled frames are assembled into a collage with the source image, and a vision-language model chooses the earliest frame that realizes the edit; this step matters because the optimal edit often occurs mid-sequence, not at the final frame.","core_discovery":"The central claim is that the natural image manifold can be traversed by a pretrained image-to-video generator, and that a frame drawn from this traversal is a better edited image than the output of a single-step image diffusion edit. Concretely, the paper proposes replacing the target prompt with a Temporal Editing Caption, a one-sentence scenario of how the edit unfolds in time; feeding this caption together with the source frame to an image-to-video model; and then selecting the earliest generated frame that satisfies the edit while staying close to the source. On TEdBench the method reports LPIPS 0.22 and CLIP-I 0.89 against the source, tied with or better than the strongest baselines, with CLIP 0.63 edit accuracy; on PosEdit it is closer to the ground-truth target than LEDITS++ on both source and target similarity. The paper also visualizes the idea by projecting generated images into a 2D PCA subspace, showing that video generation moves along a semantic path between source and target clusters while one-shot editing jumps across the manifold and drops source details.","pith_inferences":["A testable consequence the paper leaves implicit: editing quality should track video-generation benchmarks, so swapping in a better video model should improve edits with no change to the pipeline.","The temporal-caption step suggests a generative prompt strategy for any pairwise transformation task, such as image-to-image translation or style transfer, by describing the change as a short scene instead of a static description.","A natural stress test the paper only touches qualitatively is out-of-distribution edits; one can quantify how far from real-world video dynamics a caption can go before the video model stops producing a coherent path.","The early-frame selection rule encodes a prior that edits should be minimal; users who want maximal transformation could instead select the last coherent frame, making the pipeline a knob rather than a fixed method."],"forward_implications":["Edits that require structural changes, such as object pose or material transformations, can be delegated to video models instead of specialized image-editing fine-tuning.","The same pipeline, without task-specific training, extends to restoration-style tasks: deblurring, denoising, outpainting, and relighting, by choosing temporal captions that describe focus, noise removal, camera motion, or lighting change.","Frame selection becomes a user-facing control: the same generated video can serve multiple valid edits, with the chosen frame determining how far the transformation goes.","Because the source image is the first frame, identity and content preservation are enforced by construction, and later frames can be rejected to avoid drift.","The performance of the whole approach is tied to the quality of the image-to-video backbone, so improvements in video generation should transfer directly to image editing."],"supporting_citations":[{"why":"Supplies the CogVideoX image-to-video backbone that generates the temporal editing path from the source frame.","marker":"[48]"},{"why":"Defines the TEdBench benchmark and the Imagic baseline that Frame2Frame is measured against.","marker":"[22]"},{"why":"Provides LEDITS++, the strongest baseline in the human survey and quantitative comparisons.","marker":"[4]"},{"why":"Provides the SDEdit baseline for noise-injection image editing.","marker":"[31]"},{"why":"Provides the Pix2Pix-Zero baseline for zero-shot image-to-image translation.","marker":"[35]"},{"why":"Supplies the CLIP model used for the CLIP-I and CLIP edit-accuracy metrics and for the manifold visualization features.","marker":"[36]"},{"why":"Supplies UTD-MHAD, the source dataset from which the new PosEdit pose-editing benchmark is curated.","marker":"[6]"},{"why":"Inspires the collage-based frame selection, where a vision-language model reads an image grid as a proxy for video comprehension.","marker":"[23]"},{"why":"Provides FlowEdit, a recent flow-based editing baseline compared on TEdBench.","marker":"[24]"}],"fun_headline_variants":["Video generation paves a smooth path for image editing","Editing via video frames: better fidelity than one-shot diffusion","Turn edits into video transitions, keep source intact","Image manifold traversal via video beats single-step editing","Let a video model walk your edit, then pick the best frame"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that a pretrained image-to-video model, prompted only with a rewritten caption, will actually generate a temporally coherent sequence that starts at the source image and reaches the requested edit; if the backbone fails to produce that transition, neither the captions nor the frame selector can recover a good edit.","fun_headline_variants_meta":{"raw":{"variants":["Video generation paves a smooth path for image editing","Editing via video frames: better fidelity than one-shot diffusion","Turn edits into video transitions, keep source intact","Image manifold traversal via video beats single-step editing","Let a video model walk your edit, then pick the best frame"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1524,"prompt_tokens":923,"completion_tokens":601,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":523}},"tokens_in":539,"tokens_out":601,"duration_ms":6841,"temperature":1.0,"reasoning_tokens":523,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:58:04.707190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Frame2Frame on a benchmark of prompts whose edits cannot be described as plausible continuous motion, such as 'a chair sawed in half', 'a photo of an open door', or 'two bananas', and measure with CLIP and human raters whether any generated frame satisfies the target prompt while remaining close to the source. The claim predicts these still succeed because the video model can realize physical and semantic transformations; a systematic failure on such prompts, where the earliest matching frame already drifts from the source, would refute the generality of the temporal reformulation.","supporting_citations":[{"cited_title":"Imagic: Text-based real image editing with diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the TEdBench benchmark and the Imagic baseline that Frame2Frame is measured against."},{"cited_title":"Ledits++: Limitless image editing using text-to-image models","cited_arxiv_id":null,"evidence_quote":"Provides LEDITS++, the strongest baseline in the human survey and quantitative comparisons."},{"cited_title":"SDEdit: Guided image synthesis and editing with stochastic differ- ential equations","cited_arxiv_id":null,"evidence_quote":"Provides the SDEdit baseline for noise-injection image editing."},{"cited_title":"Zero-shot image-to- image translation","cited_arxiv_id":null,"evidence_quote":"Provides the Pix2Pix-Zero baseline for zero-shot image-to-image translation."},{"cited_title":"Utd- mhad: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor","cited_arxiv_id":null,"evidence_quote":"Supplies UTD-MHAD, the source dataset from which the new PosEdit pose-editing benchmark is curated."}],"review_version":1}