{"id":"133696cb-6a5b-4da5-a9ea-07d2e372f532","arxiv_id":"2411.15028","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FloAtControlNet animates clothing by flow-warping self-attention maps in a normal-map-conditioned ControlNet, improving temporal coherence and reducing background flicker.","lead":"This paper presents a training-free method that turns a text prompt and a sequence of clothing surface normal maps into a short clothing animation video. It warps self-attention features in a diffusion model using the motion estimated from the normal maps, which reduces flicker and makes fabric motion look more natural.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed 'beats all baselines' is not independently supported: the quantitative metrics and user-study question measure the same input-flow adherence that Eqs. 2-3 enforce, and the closest-baseline margins are small with no significance testing.","rationale":"I considered two candidate concerns. The first is the reader's stated weakest assumption: RAFT optical flow computed on surface normal maps, treated as RGB images, may not faithfully transfer garment motion into self-attention space. This is a genuine risk, and the paper's own limitation section admits that erroneous normal maps cause severe imperfections. However, the visible outputs and the background-suppression argument suggest the flow signal is at least usable, so I do not treat this as the single most load-bearing issue. The second concern is that the reported quantitative evidence for 'beats all baselines' is not independent of the method's mechanism. Every main metric and the user-study question measure adherence to the input normal-map flow, which is precisely the signal injected by Eqs. 2-3. The margins over the closest baseline are small, no error bars are given, alpha is tuned on the reported metrics, and the mask threshold is unspecified. These are concrete, checkable threats to the central claim. The paper is still plausible and has a clear training-free design, so I would not reject it, but the superiority claim should be conditional on a more rigorous, independent evaluation. This matches the reader's CONDITIONAL verdict, so I recommend no change.","tokens_in":14568,"tokens_out":6308,"duration_ms":66053,"concrete_test":"Run FloAtControlNet against Rerender-A-Video Adapt and FeatInControlNet+Mask on a fixed set of at least 30 normal-map sequences, using at least 5 independent latent seeds per method. Compute F-RMSE, Self-LPIPS, and user-study preference on held-out sequences, with sequence-level paired bootstrap confidence intervals (10,000 resamples). Also sweep the Eq. 4 threshold over {0.5, 1, 2, 5} pixels and alpha over {0.2, 0.4, 0.6}, selecting alpha on a validation split rather than the test set. If the 95% confidence interval for the FloAtControlNet minus Rerender-A-Video difference includes zero, or if the ranking flips for any plausible threshold or alpha, the 'beats all baselines' claim is not established. Releasing code and random seeds is necessary for this check to be performed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FloAtControlNet 'beats all baselines' rests on evidence that is partly circular with respect to the method's design. Section 4.2 defines N-RMSE, N-PSNR, F-RMSE, and F-PSNR as agreement between input normal maps/flows and normal maps/flows estimated from the generated RGB frames. But in Section 3.4, Eq. 2 and Eq. 3 directly inject the input normal-map flow into the self-attention maps and force near-zero-flow background regions to reuse the frame-0 attention output. Thus these metrics reward exactly the property the method is engineered to maximize; they are not an independent test of whether the animation is more natural. The user study in Section 4.3 similarly asks which generated video 'best resembles' the input normal-map animation, again rewarding flow adherence. Moreover, the numerical advantage over the closest baseline Rerender-A-Video Adapt is small: F-RMSE 54.887 vs. 56.873 and N-RMSE 19.396 vs. 19.456. No error bars or significance tests are reported, even though the 2990 (normal, prompt) pairs are generated from only 230 normal-map sequences, so the samples are highly correlated. The hyperparameter alpha in Eq. 2 is selected using the same Self-LPIPS/Self-SSIM metrics in Figure 6, and the mask threshold in Eq. 4 is never specified. None of this shows the method is wrong, but it means the superiority claim is not yet supported by the presented experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FloAtControlNet, a training-free method for generating clothing cinemagraphs from a text prompt and a sequence of normal maps. The method uses a normal-map-conditioned ControlNet and injects optical flow computed on the normal maps (via RAFT) into the self-attention maps: each frame's self-attention is recomputed as a linear combination of its own attention and the flow-warped attention from the previous frame (Eq. 2), and a binary flow mask forces zero-flow regions to reuse the previous frame's attention (Eqs. 3-4). The authors claim this improves clothing motion naturalness and suppresses background flicker, and they report quantitative comparisons on N-RMSE/N-PSNR/F-RMSE/F-PSNR, Self-LPIPS/SSIM, ablations, and a user study, concluding that the method beats all baselines.","tokens_in":14900,"tokens_out":5618,"duration_ms":51751,"significance":"If the empirical claims are supported, the method is a simple, training-free alternative for controllable clothing animation that exploits normal-map flow to guide self-attention warping. The ablation demonstrating the individual contributions of mask correction and flow warping is useful, and the qualitative results show plausible outputs with reduced background flicker. However, the evaluation relies on metrics that reward the exact properties the method is designed to enforce, the reported gains over the closest baseline are small and lack significance testing, and a key hyperparameter (the mask threshold) is not disclosed. The significance of the claimed superiority is therefore not yet established, though the core idea is coherent and potentially valuable.","major_comments":[{"comment":"The metrics N-RMSE, N-PSNR, F-RMSE, and F-PSNR are computed as agreement between the input normal maps/flows and normal maps/flows estimated from the generated RGB frames. The method directly injects the input normal-map flow into the self-attention maps (Eq. 2) and uses the flow mask to freeze zero-flow background regions (Eq. 3). These metrics therefore reward exactly the property the method is engineered to maximize; they do not independently establish that the animation is more natural or of higher visual quality. Moreover, the numerical advantage over the closest baseline, Rerender-A-Video Adapt, is small (N-RMSE 19.396 vs. 19.456; F-RMSE 54.887 vs. 56.873) and no error bars or significance tests are provided, even though the 2990 (normal, prompt) pairs are generated from only 230 normal-map sequences and are thus highly correlated. The claim that the method \"beats all baselines\" is not supported by the experiments as presented.","section":"Section 4.3, User Study"},{"comment":"The user study asks participants which generated video's clothing-region animation \"best resembles\" the input normal-map animation. This is the same fidelity criterion that Eqs. 2-3 optimize, so the study does not assess naturalness or visual quality independently. The study uses only 10 examples, 67 users, and no statistical test is reported; the observed preference counts (41.3% vs. 31.3% and 27.3%) could easily arise from chance. Please add a preference test that asks about plausibility or quality without showing the normal-map input, and report confidence intervals or a significance test.","section":"Section 4.3"},{"comment":"The background-suppression effect is enforced by construction. In background regions where the optical flow is zero, the mask M_fc is zero, so Eq. 3 reduces to A^i_tcor = A^{i-1}_t; by induction, the self-attention in zero-flow regions is frozen across all frames. The reported reduction in background flicker is therefore an expected consequence of the design rather than an empirical discovery. The paper should state this explicitly and frame the qualitative background comparisons as verification of the implementation, not as evidence of a novel capability.","section":"Section 3.4"},{"comment":"The threshold in Eq. 4 is never specified in the paper or the supplementary material, which is a reproducibility gap. Additionally, the mixing weight alpha is selected by an ablation (Figure 6) using the same Self-LPIPS and Self-SSIM metrics that are later used to claim superiority in Table 3, creating a selection-on-the-evaluation-metric concern. Please provide the exact threshold, report sensitivity to both alpha and the threshold, and avoid choosing hyperparameters on the same metrics used for the main comparison.","section":"Section 3.4"},{"comment":"The paper acknowledges that erroneous normal maps can cause severe imperfections, but it does not quantify robustness to normal-map noise or error. Since the flow is computed on the normal maps and then used to warp attention, the method's core assumption is that the normal-map flow is a faithful proxy for true garment motion. Please include experiments with corrupted or perturbed normal maps to show when the method degrades and to what extent.","section":"Section 5"}],"minor_comments":[{"comment":"There are inconsistent spellings: \"FloAtControlNet\" appears as \"FloatControlNet\" and \"Rerender-A-Video\" as \"Renderer-A-Video\" in the same section. Please standardize the method and baseline names throughout.","section":"Section 4.3"},{"comment":"In Table 2, CycleNet Reshading reports Self-LPIPS of 0.002, which is lower (better) than FloAtControlNet's 0.025. The text explains this as due to static outputs, but the abstract's broad claim that the method \"beats all baselines\" should be qualified, since CycleNet is not included in the ablation table where \"beats all baselines\" is claimed for Self-LPIPS/SSIM.","section":"Table 2"},{"comment":"The loop over s = 0 to 319 is presumably the channel dimension of the self-attention output (1x64x64x320), but this is not stated. Please clarify the notation so that the algorithm is self-contained.","section":"Algorithm 1"},{"comment":"The binary mask in Eq. 4 uses a hard threshold with no smoothing or dilation, which may create spatial discontinuities in the attention maps. A brief discussion of the threshold choice and its sensitivity would help readers understand the practical behavior.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The core concern is that the headline quantitative and user-study evidence is circular with respect to the method's design, and the margins over the strongest baseline are small without significance testing. I would request the authors to substantially strengthen the evaluation before publication. The manuscript also omits a key hyperparameter (the mask threshold), which should be fixed during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new piece here is using optical flow computed on normal maps to warp self-attention maps in a ControlNet pipeline, rather than warping latents or RGB features. That lets the method work without any input video, and the background suppression is neatly designed: zero-flow regions inherit the previous frame's attention, so the background is frozen by construction. The paper also does some things well: the attention visualizations in Fig. 3 give a concrete reason for the flow warping, the ablations isolate the mask and warping contributions, and the limitations section is unusually honest about normal-map sensitivity and the 64×64 attention resolution.\n\nThat said, the central claim that the method \"beats all baselines\" is not yet supported. The quantitative metrics in Table 1 measure agreement between input normal maps (and their flows) and normal maps estimated from the generated RGB frames. That is exactly the quantity Eqs. 2-3 are engineered to maximize, so the comparison is partly circular. The user study asks which video \"best resembles\" the input normal-map animation, again rewarding flow adherence rather than perceived naturalness. The margins over the closest baseline are small — N-RMSE 19.396 vs 19.456, F-RMSE 54.887 vs 56.873 — with no error bars, and the 2990 (normal, prompt) pairs come from only 230 normal-map sequences, so the samples are highly correlated. Alpha is also tuned on the same Self-LPIPS/Self-SSIM metrics that are later reported as evidence, and the mask threshold in Eq. 4 is never specified. None of this is fatal; it just means the superiority claim requires a firmer evaluation before I'd trust it.\n\nI'd also want to see code or more implementation details, since a training-free method like this should be easy to reproduce once the mask threshold and layer choice are pinned down. The paper's decision not to compare with TokenFlow is reasonable given the DDIM inversion issue, and they do show TokenFlow examples in the supplement, which is more than most papers do.\n\nThis paper is for people working on training-free video generation, controllable diffusion, or garment animation. It deserves a serious referee, but the referee should ask for a cleaner evaluation protocol: independent perceptual judgments, significance testing, and ideally a comparison that does not reward the method's built-in behavior. I'd accept it with major revision, not desk-reject it.","headline":"A plausible training-free recipe for clothing cinemagraphs that warps self-attention with normal-map flow; the core idea is new, but the evaluation overclaims because the metrics reward what the method is built to do.","tokens_in":15442,"tokens_out":1330,"would_cite":false,"duration_ms":14753,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free diffusion method animates clothing by warping self-attention along normal-map flow.","keywords":["clothing animation","cinemagraph generation","self-attention warping","normal map flow","optical flow","ControlNet","training-free diffusion","text-conditioned video"],"falsifier":"Run FloAtControlNet on a constant normal-map sequence, so the computed flow is zero everywhere. The method's mask-correction step should then force every frame's attention map to equal the first frame's, making the generated video exactly static; if any frame moves or flickers, the background-suppression mechanism is incomplete.","tokens_in":14385,"feed_emoji":"👗","tokens_out":7064,"duration_ms":65309,"temperature":0.7,"pith_summary":"The paper aims to show that clothing cinemagraphs can be generated from a text prompt plus a sequence of surface normal maps without any additional training. Its central move is to compute the optical flow between consecutive normal maps and use it to warp the self-attention map of the previous generated frame, blending the warped map with the current frame's map at each denoising step. The paper argues that this makes garment motion more natural, keeps high-frequency textures such as stripes and prints intact, and suppresses background flicker because the background has zero flow and therefore inherits the first frame's attention. It reports that the method outperforms the considered baselines on normal-map fidelity metrics and in a user study, while noting that standard perceptual metrics are not sensitive to subtle clothing motion.","feed_headline":"Flow-warped attention animates clothing without background flicker","feed_subtitle":"Normal-map flow guides self-attention so fabric textures stay stable while the background stays still.","key_machinery":"The load-bearing mechanism is flow-warped self-attention. After cross-frame self-attention feature injection from an anchor frame and the previous frame, the attention map of the last self-attention layer in the third up-block is blended with the previous frame's attention map warped by RAFT optical flow computed on the normal maps, and then corrected by a binary flow mask. In regions where the normal-map flow is below threshold, the mask is zero, so the corrected attention map is exactly the previous frame's map; by induction, background attention stays identical to frame 0, which the paper identifies as the source of background suppression.","core_discovery":"In the paper's framing, the discovery is that the animation of a garment is already contained in the flow field of its normal maps, and injecting this flow into the self-attention layers of a normal-map-conditioned diffusion model transfers that motion to the RGB output. For each frame and denoising step, the self-attention map is recomputed as $\\widehat{A}^i_t = \\alpha A^i_t + (1-\\alpha)\\,\\mathrm{warp}(A^{i-1}_t, f^i_c)$, then corrected with a binary mask derived by thresholding the normal-map flow so that no-flow regions reuse the previous frame's attention. Because background normals do not move, the mask forces background attention to remain equal to that of the first frame, effectively freezing the background. The paper presents this as a training-free way to obtain temporally coherent clothing animation, including high-frequency textured garments where a GAN-based predecessor fails.","pith_inferences":["Not stated in the paper: the same flow-warping recipe should transfer to other ControlNet condition modalities whose input sequences carry geometric motion, such as depth or edge maps, giving a general training-free recipe for condition-driven animation.","The background-freezing guarantee is conditional on the input normal maps having zero flow in the background; supplying normal maps with camera motion would violate the mask assumption and would test whether the method distinguishes garment motion from scene motion.","An extension implicit in the applications discussion: chaining a single-image normal-map predictor with FloAtControlNet would turn a still photograph of a person into a clothing cinemagraph, since the paper already demonstrates that predicted normal-map sequences can drive the method."],"forward_implications":["If the central claim holds, clothing cinemagraphs can be produced from a text prompt plus a simulated or predicted normal-map sequence without fine-tuning any diffusion model.","High-frequency textures such as stripes, leopard print, tie-dye, and lace can be animated naturally, a case where the GAN-based predecessor is reported to fail.","Because background attention is pinned to the first frame wherever flow is zero, the method suppresses background flicker at the cost of freezing all zero-flow regions.","Standard perceptual metrics such as LPIPS, SSIM, and CLIP under-report quality differences for subtle clothing motion, so normal-map conditioning metrics and user studies become the informative comparisons."],"supporting_citations":[{"why":"Supplies the normal-map-conditioned ControlNet backbone on which every generated frame is built.","marker":"[1]"},{"why":"Contributes the cross-frame self-attention feature injection that the method extends with flow warping.","marker":"[8]"},{"why":"Computes the optical flow on normal maps that warps the self-attention maps.","marker":"[24]"},{"why":"Defines the normal-map-sequence representation of garment motion and provides a strong GAN-based baseline and input data.","marker":"[7]"},{"why":"Provides the latent-warping baseline that the paper argues is beaten by warping in self-attention space.","marker":"[17]"},{"why":"ControlNet-based video baseline adapted for comparison in the experiments.","marker":"[27]"},{"why":"Supplies the wind-simulated cloth normal-map sequences used for the dataset.","marker":"[26]"},{"why":"Estimates normal maps from generated RGB frames for the N-RMSE and N-PSNR fidelity metrics.","marker":"[28]"}],"fun_headline_variants":["Training-free flow-warped attention freezes background, moves fabric","Normal-map flow guides self-attention to animate clothing naturally","Flow-warping self-attention avoids background flicker in cloth animation","Fabric motions follow normal-map flow; background stays frozen in place"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the movement field estimated from the normal-map sequence, computed by treating normal maps as ordinary images, faithfully describes how the fabric really moves; if that estimate is wrong, warping attention along it will distort the generated clothing.","fun_headline_variants_meta":{"raw":{"variants":["Training-free flow-warped attention freezes background, moves fabric","Normal-map flow guides self-attention to animate clothing naturally","Flow-warping self-attention avoids background flicker in cloth animation","Fabric motions follow normal-map flow; background stays frozen in place"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1450,"prompt_tokens":1050,"completion_tokens":400,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":326}},"tokens_in":666,"tokens_out":400,"duration_ms":4352,"temperature":1.0,"reasoning_tokens":326,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:35:00.355324+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FloAtControlNet on a constant normal-map sequence, so the computed flow is zero everywhere. The method's mask-correction step should then force every frame's attention map to equal the first frame's, making the generated video exactly static; if any frame moves or flickers, the background-suppression mechanism is incomplete.","supporting_citations":[{"cited_title":"Raft: Recurrent all-pairs field transforms for optical flow","cited_arxiv_id":null,"evidence_quote":"Computes the optical flow on normal maps that warps the self-attention maps."},{"cited_title":"Blowing in the wind: Cyclenet for human cinemagraphs from still images","cited_arxiv_id":null,"evidence_quote":"Defines the normal-map-sequence representation of garment motion and provides a strong GAN-based baseline and input data."},{"cited_title":"Estimating and exploiting the aleatoric uncertainty in surface normal estimation","cited_arxiv_id":null,"evidence_quote":"Estimates normal maps from generated RGB frames for the N-RMSE and N-PSNR fidelity metrics."}],"review_version":1}