{"id":"e4f2ff9e-5f86-49e2-8ac3-b13972c33f48","arxiv_id":"2412.11710","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ReAtCo improves text-guided video editing by using attention-map gradients to place edited objects in user-specified regions and by re-injecting the original background during diffusion sampling.","lead":"Researchers present ReAtCo, a training-free method to make text-guided video editing more controllable, so that objects appear in the correct positions and the background stays intact. It works by nudging the diffusion model's attention maps during generation and by fusing the original background back into each denoising step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"IRJS Eq. (8) claims the stitched latent is in-distribution, but the covariance of the masked Gaussian mixture does not match the denoiser posterior unless the posterior variance equals the forward marginal variance, so the 'eX ~ N(µθ,Σθ)' derivation is invalid.","rationale":"The central claim of improved controllability is supported by the RAD ablation and qualitative results; the VISOR jump and attention-map visualization give real evidence for the attention-refocusing mechanism. The weakest point is indeed IRJS's distributional premise, and the reader identified it correctly. Under the paper's own Gaussian notation, the masked mixture of two independent draws from different covariances is not, in general, distributed as either component. This is not merely a missing proof; the equality fails for the actual posterior covariances used by DDPM/DDIM samplers, so the derivation in the paragraph after Eq. (8) is mathematically unsupported. The K=20% hyperparameter selection is a milder optimistic-bias issue, and the lack of error bars is a reporting gap, but neither is as load-bearing as the IRJS in-distribution step because the latter underpins the invariant-region preservation claim. Since the empirical ablation shows IRJS improves PSNR/LPIPS, the concern is a validity gap rather than a demonstrated failure; the verdict should remain CONDITIONAL pending the proposed test or a revised derivation.","tokens_in":15091,"tokens_out":8377,"duration_ms":85117,"concrete_test":"Compute the KL divergence (or Fréchet distance) between eX(t-1) from Eq. (8) and the Gaussian transition N(µθ(X(t),t),Σθ(X(t),t)) at representative timesteps (e.g., t=1000, 500, 50) using the released code and Stable Diffusion v1.4 under the exact sampler (DDIM eta=0) used in the paper. If the KL is large at early timesteps, the in-distribution claim fails. Then rerun editing with eX(t-1) replaced by a sample from N(µθ,Σθ) (i.e., add posterior noise instead of injecting the diffused source) and compare PSNR/LPIPS and VISOR. If the distribution-preserving replacement matches IRJS, the benefit is not from source injection; if it degrades, IRJS works for another reason and the derivation should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing component is IRJS: the paper's claim that it 'mitigates sampling errors' relies on Eq. (8) and the assertion that eX(t-1)=X(t-1)M+V(t-1)(1-M) is 'abided by the distribution of X(t-1)'. This assertion is not derived and is generally false. Let X~N(µθ,Σθ) and V~N(sqrt(αbar_{t-1})V0,(1-αbar_{t-1})I) be independent, as the paper states. For fixed binary mask M, the covariance of eX is MΣθM^T + (1-M)(1-αbar_{t-1})I(1-M)^T, which equals Σθ only when Σθ=(1-αbar_{t-1})I. In DDPM the posterior variance Σθ is β̃_t I, which is smaller than the forward marginal variance; in DDIM with eta=0 the transition is deterministic, so the variance is zero. Thus eX is not a draw from the same Gaussian transition, and the noise level jumps at the mask boundary. The paper's 'we can derive' is therefore a claim-without-derivation, not a proof. The empirical PSNR/LPIPS gains in Tab. 3 show the injection helps, but they do not establish the distributional premise; if the mixture is out-of-distribution, IRJS may work only because the UNet tolerates the discontinuity, so the stated mechanism is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReAtCo, a training-free inference-time method for text-guided video editing built on Tune-A-Video. It introduces two components: Re-Attentional Diffusion (RAD), which uses gradient updates on the noisy latent to maximize cross-attention responses of selected words inside user-specified masks and suppress them outside, and Invariant Region-guided Joint Sampling (IRJS), which stitches the diffused source video's invariant region into the denoised latent at each step. Experiments on LOVEU-TGVE-2023 and additional samples compare ReAtCo against Tune-A-Video, FateZero, StableVideo, TCVE, and TokenFlow, reporting a large VISOR gain and modest improvements in Frame Consistency, Textual Alignment, and PickScore. The paper also provides ablations, cross-attention visualizations, and a sensitivity study of the top-k hyperparameter K.","tokens_in":15336,"tokens_out":4998,"duration_ms":45441,"significance":"If the results hold, RAD provides a practical, plug-and-play spatial control for video editing without retraining, and the release of code supports reproducibility. The ablation study and cross-attention visualizations give some evidence for RAD's effectiveness. However, the distributional claim underlying IRJS is unsupported, the headline VISOR metric is closely aligned with the RAD objective and its hyperparameter is selected on the evaluation set, and the quantitative comparisons lack variance estimates. These issues currently temper the significance of the paper's central claims, although the core idea is promising and the limitations appear addressable.","major_comments":[{"comment":"In the paragraph immediately after Eq. (8), the paper claims that because X(t-1) ~ N(µθ, Σθ) and V(t-1) ~ N(√ᾱ_t V(0), (1-ᾱ_t)I) are 'approximately the same distribution,' the masked mixture eX(t-1) = X(t-1)M + V(t-1)(1-M) is 'abided by the distribution of X(t-1).' This is not a derivation. For a fixed binary mask M, the covariance of eX(t-1) is MΣθM^T + (1-M)(1-ᾱ_t)I(1-M)^T, which equals Σθ only in the special case Σθ = (1-ᾱ_t)I. In DDPM the posterior variance is β̃_t I, which is smaller than the forward marginal variance, and in DDIM with η=0 the transition is deterministic, so the mixture is generally not a draw from N(µθ, Σθ) and the noise level changes discontinuously at mask boundaries. The Appendix's PSNR/LPIPS improvements show that the injection helps empirically, but they do not validate the distributional premise. Since IRJS is a core contribution, the authors should either provide a correct distributional analysis (e.g., showing the mismatch is negligible at the resolutions and timesteps used) or explicitly reframe IRJS as an empirical heuristic and quantify boundary artifacts.","section":"Invariant Region-guided Joint Sampling (Eq. 8)"},{"comment":"The value K=20% is selected by maximizing VISOR on the evaluation set. This is problematic because the RAD objective directly raises cross-attention responses inside the masks and suppresses them outside, while VISOR measures exactly whether objects appear in the expected spatial relation (left/right/above/below) derived from those masks. Part of the reported VISOR gain (70.62 vs. 25.31 in Table 1) is therefore by construction rather than an independent measure of editing quality. The authors should report how K was chosen (e.g., on a validation split), give VISOR for K values not selected, and ideally also report an object-detection-based metric that is not aligned with the loss. Without this, the headline quantitative claim is overstated.","section":"Exploring the effective K in topk(·, K) (Fig. 7)"},{"comment":"Tables 1 and 2 report point estimates only, with no error bars, significance tests, or number of videos per condition. The text states that ReAtCo 'consistently improves' and 'achieves superior video editing performance,' but the margins on Frame Consistency (95.24 vs. 94.79 for TCVE) and PickScore (20.70 vs. 20.58) are small relative to typical run-to-run variation for diffusion-based editing. Please report standard deviations over multiple runs or a paired per-video test, and state how many videos and prompts contribute to each number. This is necessary to support the 'consistently' claim.","section":"Quantitative results (Tables 1 and 2)"},{"comment":"Table 3 reports PSNR/LPIPS for 'Ours w/o IRJS' vs. 'Ours.' Because IRJS literally replaces the invariant region with the diffused source video sample V(t-1), which is an approximation of the original source content, it is expected by construction that PSNR rises and LPIPS falls. This does not by itself demonstrate that the generated object region is 'harmonized' with the invariant region or that border artifacts are reduced, which is the stated motivation of IRJS. Please add a quantitative measure of boundary artifacts or a user study, or at least show cropped boundary regions for Ours w/o IRJS and Ours. The single frame in Fig. 3(d) is not sufficient.","section":"Appendix: Ablation Study of IRJS (Table 3)"}],"minor_comments":[{"comment":"The symbol V(t-1) in Eq. (8) conflicts with V used for the source video in the problem statement; please use a distinct notation such as V_src(t-1) or Z(t-1).","section":"Notation (Eq. 8)"},{"comment":"The action of topk(·, K) is used in the equations but not formally defined; as written, it is ambiguous whether it returns the set of K largest values or their sum. Please define it explicitly.","section":"Eqs. (3)-(5)"},{"comment":"The paper calls the method 'training-free,' but RAD requires backpropagation through the UNet at inference; please clarify that no parameter updates are performed and that the gradient is used only to update the latent.","section":"Terminology"},{"comment":"The statement that 'αt decays linearly from 1 to 0.5 during the denoising process' should specify the timestep range over which this decay occurs, given that the model uses 50 denoising steps.","section":"Appendix: More Implementation Details"},{"comment":"The row with '× ×' corresponds to the Tune-A-Video baseline; please state this explicitly in the caption for clarity.","section":"Table 2 caption"},{"comment":"There are minor typographical errors: 'Co ntrollable' in the Abstract and 'editied' in the Introduction should be corrected.","section":"Introduction and Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for a computer vision venue, and the release of code is a strength. The main risk is the unsupported IRJS distributional claim, which the authors could address by reframing IRJS as an empirical heuristic or by providing a correct analysis of the variance mismatch. The VISOR-selection circularity and lack of error bars are also important but appear fixable. I would not reject on the basis of the DDIM/DDPM mismatch alone, since the empirical component may survive, but the current text overclaims a derivation and the headline metric needs more careful reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nShort version: the method works better than the theory section deserves. ReAtCo combines a training-free cross-attention gradient update (RAD) with masked latent injection (IRJS) for text-guided video editing. On the LOVEU-TGVE benchmark it takes VISOR from 25.31 (best baseline) to 70.62, and the ablations convincingly attribute that jump to RAD. The cross-attention visualizations show the refocusing behaving as intended. The code is released, and there is an honest limitations section that acknowledges temporal unsmoothness and the memory cost of gradient tracking. That's a real contribution, not a trophy.\n\nThe main soft spot is Eq. (8). The claim that the stitched latent eX(t-1) is 'abided by the distribution of X(t-1)' does not follow. The covariance of the masked mixture of X and V is M*Sigma*M^T + (1-M)*(1-alpha_bar_t)*I, which equals Sigma only if Sigma = (1-alpha_bar_t)*I. In DDPM the posterior variance is beta_tilde_t*I, smaller than the forward marginal; in DDIM with eta=0 it is zero. So the derivation is wrong as written. The empirical PSNR/LPIPS gains show IRJS helps in practice, but it should be presented as a heuristic latent replacement, not a distribution-preserving operation. That is a fixable but load-bearing flaw: the paper's stated mechanism for IRJS is unsupported.\n\nSecond, the K=20% ablation (Fig. 7) selects the hyperparameter by maximizing VISOR on the evaluation set, and RAD directly optimizes spatial attention inside the masks that VISOR tests. So part of the VISOR gain is by construction. The other three metrics improve by much smaller margins, and there are no error bars or significance tests. 'Consistently improves' oversells the numbers.\n\nThird, novelty is incremental: RAD is a close relative of attention-refocusing methods like Grounded T2I and ReCO, ported to video with a top-k formulation, and IRJS resembles known latent-blending background preservation. The combination is new and useful, but the conceptual step is not large.\n\nAll that said, I would not desk-reject this. The core mechanism is clear, the ablations are honest, and the code is available. A serious referee should ask the authors to correct the Eq. (8) claim, add error bars and dataset sizes, and temper the 'consistent' language. If they fix those, it's a solid paper for the video-editing community. I'd bring it to a reading group as an example of a training-free editing trick that works despite a shaky theoretical justification.\n\nRecommendation: send to peer review, with a request for revision.","headline":"Training-free attention-refocusing for video editing works better than its theory; the VISOR leap is real but partly by construction, and the IRJS distribution claim is wrong as written.","tokens_in":15954,"tokens_out":2962,"would_cite":true,"duration_ms":27020,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims spatial control in text-guided video editing can be achieved without retraining by steering cross-attention maps and re-injecting the original unchanged regions.","keywords":["text-guided video editing","cross-attention control","training-free controllable generation","diffusion models","attention refocusing","invariant region preservation","spatial relationship control","video diffusion editing"],"falsifier":"Compute the edited video for a prompt identical to the source prompt under IRJS; if the output deviates measurably from the source video in the invariant region or drifts frame-to-frame, the stitched latent is not behaving like a valid sample from the transition distribution. A second direct test is to compare IRJS against simply decoding the generated latent and compositing the clean source background, using PSNR and LPIPS on the invariant region; comparable scores would show that per-step injection, not final compositing, is responsible for the reported fidelity gains.","tokens_in":1726,"feed_emoji":"🎬","tokens_out":7182,"duration_ms":91025,"temperature":0.7,"pith_summary":"ReAtCo argues that the two persistent failures of prompt-based video editing—objects appearing in the wrong place or in the wrong number—can be fixed without any training, by manipulating the cross-attention maps that connect each word to pixels. During each denoising step it defines a loss that rewards high attention inside user-provided masks and low attention outside them, then takes a small gradient step on the noisy video latent. To keep untouched regions faithful, it stitches the edited latent inside the masks with the diffused source video outside them at every timestep. If correct, this gives a plug-and-play layer of spatial control that works on top of existing video editing models such as Tune-A-Video.","feed_headline":"Nudging attention maps puts edited objects where users want","feed_subtitle":"Training-free method fixes misplaced objects and wrong counts by steering attention and injecting the source background","key_machinery":"The load-bearing mechanism is the cross-attention map between text and video frame in the latent diffusion backbone, $A_i^{(t)} = \\mathrm{Softmax}(Q_i^{(t)}K^\\top/\\sqrt{d}) \\in \\mathbb{R}^{L\\times H\\times W}$, which records how strongly each word attends to each pixel region. RAD turns each word's attention map into a differentiable objective: maximize the top-k activations inside the user's mask and minimize the top-k activations outside it, then back-propagate through the denoiser to update the noisy latent. IRJS is the second mechanism: at each step it replaces the latent outside the object masks with the source video's own noised latent, so the invariant region is regenerated from the original content rather than sampled freely.","core_discovery":"The central claim is that controllability of text-guided video editing, specifically the spatial location and number of multiple foreground objects, can be achieved by refocusing cross-attention activation responses during denoising rather than by retraining or finetuning. The paper proposes Re-Attentional Diffusion (RAD), which defines inner-region and outer-region constraints on the cross-attention maps of the words of interest, uses top-k responses to stay robust to imprecise masks, and updates the noisy sample as $X^{(t)} \\leftarrow X^{(t)} - \\alpha_t \\nabla(\\mathcal{L}_{\\mathrm{IR}}(t)+\\mathcal{L}_{\\mathrm{OR}}(t))$. It further claims that an Invariant Region-guided Joint Sampling (IRJS) strategy removes sampling drift in unchanged regions by forming $\\tilde{X}^{(t-1)} = X^{(t-1)}\\times M + V^{(t-1)}\\times (1-M)$, arguing that this mixture still follows the denoiser's Gaussian transition distribution. On the evaluated benchmarks ReAtCo reports the best scores in frame consistency, textual alignment, PickScore, and especially VISOR, with the spatial-relationship metric rising from 25.31 for the strongest baseline to 70.62.","pith_inferences":["The same top-k attention loss could be applied per object to enforce counts, for example by penalizing attention mass outside a single connected mask per word, a setting the paper does not test.","Because IRJS injects the source video's noised latent outside masks, the quality ceiling of the invariant region is set by DDIM inversion; a source video that inverts poorly should show background artifacts even with IRJS.","RAD's inner and outer loss could transfer directly to text-to-image generation, since they only assume a backbone with cross-attention maps; the paper demonstrates them only for video editing."],"forward_implications":["Spatial control becomes a training-free add-on: any video diffusion editing model built on cross-attention can inherit location control by injecting masks and gradient updates at denoising time.","Multi-object edits with explicit left, middle, and right relations become feasible, as demonstrated with two-object and three-object scenes in the paper.","Untouched regions stay faithful: the appendix reports that IRJS raises invariant-region PSNR from 29.29 to 36.57 dB and lowers LPIPS from 0.2706 to 0.0206.","The method needs only bounding-box style masks from the user, not per-video training or finetuning.","The attention maps become inspectable: the paper shows that with RAD, the jellyfish word's attention converges into the user-specified region as denoising progresses, while without RAD it drifts away."],"supporting_citations":[{"why":"Supplies Tune-A-Video, the off-the-shelf video diffusion editing model used as the backbone for all ReAtCo experiments.","marker":"(Wu et al. 2023a)"},{"why":"Supplies the Stable Diffusion latent diffusion model whose cross-attention maps RAD modifies.","marker":"(Rombach et al. 2022)"},{"why":"Supplies DDIM inversion, used to convert the source video into noise before editing.","marker":"(Song, Meng, and Ermon 2021)"},{"why":"Supplies the diffusion sampling formulation and noise scheduler that IRJS relies on for its Gaussian transition argument.","marker":"(Ho, Jain, and Abbeel 2020)"},{"why":"Establishes cross-attention manipulation as a viable control mechanism and informs the choice of attention resolution used by RAD.","marker":"(Hertz et al. 2023)"},{"why":"Provides the sampling-drift observation that motivates IRJS's per-step injection of original invariant-region content.","marker":"(Daras et al. 2024)"},{"why":"Supplies the VISOR metric used to measure spatial relationship correctness between edited objects.","marker":"(Gokhale et al. 2022)"},{"why":"Supplies the PickScore metric used to measure human preference alignment of edited frames.","marker":"(Kirstain et al. 2023)"},{"why":"Supplies LPIPS, used in the appendix to quantify invariant-region reconstruction fidelity.","marker":"(Zhang et al. 2018)"}],"fun_headline_variants":["Steering attention maps fixes object placement in video edits","Training-free video editing that places objects accurately","Attention refocusing controls object location and count in videos","ReAtCo: nudge attention to fix misplaced objects and wrong counts"],"cache_read_input_tokens":17920,"weakest_assumption_plain":"IRJS assumes that a latent formed by stitching two independent noisy samples, the edited object region and the diffused source background, is still a valid sample from the denoiser's next-step Gaussian distribution, even though no proof is given that the mixture is in-distribution.","fun_headline_variants_meta":{"raw":{"variants":["Steering attention maps fixes object placement in video edits","Training-free video editing that places objects accurately","Attention refocusing controls object location and count in videos","ReAtCo: nudge attention to fix misplaced objects and wrong counts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000298,"raw_usage":{"total_tokens":1771,"prompt_tokens":1040,"completion_tokens":731,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":665}},"tokens_in":656,"tokens_out":731,"duration_ms":7024,"temperature":1.0,"reasoning_tokens":665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:39:17.150088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the edited video for a prompt identical to the source prompt under IRJS; if the output deviates measurably from the source video in the invariant region or drifts frame-to-frame, the stitched latent is not behaving like a valid sample from the transition distribution. A second direct test is to compare IRJS against simply decoding the generated latent and compositing the clean source background, using PSNR and LPIPS on the invariant region; comparable scores would show that per-step injection, not final compositing, is responsible for the reported fidelity gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes cross-attention manipulation as a viable control mechanism and informs the choice of attention resolution used by RAD."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the sampling-drift observation that motivates IRJS's per-step injection of original invariant-region content."}],"review_version":1}