{"id":"c16a910b-326a-40bb-9201-79c3011dfc3e","arxiv_id":"2412.02099","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper reports a training-free extrapolation method that combines per-patch prompt selection from cross-attention masks, ControlNet edge guidance, and permuted dilated sampling to reduce repetition and distortion at 4x to 16x upscales.","lead":"This paper presents AccDiffusion v2, a training-free recipe for making Stable Diffusion draw much larger images than it was trained for. It assigns each image patch its own text prompt, guides the denoising with edge sketches via ControlNet, and permutes global views during sampling to stabilize whole-image semantics.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eqs. 16–17 define a no-op: the random permutation of dilated samples is undone by its inverse after per-sample denoising, so 'window interaction' cannot produce the smoother global semantics shown in Fig. 7; the paper's third core contribution is unsupported as written.","rationale":"The reader's weakest_assumption—that the low-resolution draft is a faithful blueprint—is a real limitation and is candidly acknowledged in Sec. 6, but it is a design choice shared by most progressive patch-wise extrapolation methods and not an internal inconsistency. The sharper defect is the window-interaction module. For the central claim to hold, all three named mechanisms must be active. Eq. (16) and Eq. (17) are presented as coupling the dilated samples, yet no cross-sample term exists in the denoising equation; per-sample application of a pretrained denoiser commutes with relabeling the samples. This is a mathematical identity, so the claimed smoothing effect in Fig. 7 cannot be caused by the permutation. The ablations in §5.5.1 remove the entire dilated-sampling block rather than isolating the interaction, so they do not establish the contribution either. Because the patch-prompt and ControlNet modules are concretely specified and plausibly effective, a conditional revision—add the matched-seed comparison or remove the interaction claim—is appropriate; hence the reader's CONDITIONAL verdict remains unchanged. I partially agree with the reader: the blueprint concern is secondary, although the reader's rationale also correctly flagged the inert permutation.","tokens_in":17743,"tokens_out":6720,"duration_ms":65079,"concrete_test":"Run the released code on a fixed set of, say, 10 prompts at 4× resolution with fixed seeds, comparing the full pipeline against the same pipeline with the permutation in Eq. (16) replaced by the identity map and the inverse/recover step in Eq. (17) removed. If the outputs are identical up to floating-point nondeterminism, the window interaction has no effect and Fig. 7 must be rerun under matched seeds; if they differ, inspect the implementation for an undocumented cross-sample coupling, because that coupling, not the permutation, would be the active mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing defect is in §4.3. Eq. (16) applies a position-wise bijection f^h,w_t to the dilated samples D^k_t, and Eq. (17) applies its inverse after the denoising step. Because each D^k_t is denoised independently by the same pretrained model—no coupling term, no attention across samples, and no window mixing is introduced—the composition is exactly the identity. Concretely, after the permutation slot k contains D^{f(k)}; denoising gives the denoised version of D^{f(k)}; the inverse mapping in Eq. (17) returns the denoised version of D^k to slot k. Thus the operation has no effect on the latent variables. The qualitative comparison in Fig. 7(a) vs 7(b) is not matched-seed controlled, and no quantitative metric accompanies it; any visible difference must come from initialization noise or from implementation behavior not specified in the paper. Since the abstract and conclusion credit 'dilated sampling with window interaction' with suppressing repetition and distortion, the central causal claim about this component is currently unsupported. This does not by itself refute the patch-prompt or ControlNet contributions, but it means the full AccDiffusion v2 pipeline's advantage over its predecessor cannot be attributed to the permutation module as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AccDiffusion v2, a training-free patch-wise method for extrapolating pre-trained latent diffusion models (SDXL) to higher resolutions (up to 16x). It introduces three components: (i) patch-content-aware prompts derived from low-resolution cross-attention maps, (ii) ControlNet injection of canny edges from the upscaled low-resolution draft to reduce local distortion, and (iii) dilated sampling with a position-wise bijection (window interaction) intended to make global semantic information smoother. Experiments compare against eight training-free baselines at 4x, 9x, and 16x resolutions using FID, IS, and CLIP score, with qualitative comparisons and ablations.","tokens_in":17914,"tokens_out":5610,"duration_ms":51292,"significance":"If validated, the patch-content-aware prompt and ControlNet-structure contributions would be useful practical extensions for training-free high-resolution image generation on consumer GPUs. The code is released, the comparisons are external to eight baselines, and the paper includes ablations of each module. However, the third contribution as described in §4.3 is mathematically a no-op, and the quantitative evidence for state-of-the-art performance is thin without variance or significance testing. The main value lies in the prompt decoupling idea and the ControlNet local-structure prior, both of which are plausible and partially supported by the ablations.","major_comments":[{"comment":"Equations (16) and (17) define a position-wise bijection f applied to the dilated samples before denoising and its inverse after denoising. Since each dilated sample is denoised independently by the same pretrained network and the paper introduces no cross-sample coupling, attention, or mixing between samples, the composition f^{-1}∘f is the identity and the operation has no effect on the latent variables. The claim that this 'window interaction' produces smoother global semantics (Fig. 7) is therefore unsupported as written; Fig. 7(a) versus 7(b) is a non-seed-matched qualitative pair with no quantitative metric. Please specify the actual coupling mechanism (e.g., mixing latents within the 2x2 windows after denoising) and provide matched-seed or metric-based evidence, or remove this contribution from the claims.","section":"4.3"},{"comment":"The ControlNet condition in Eq. (13) is built from canny edges of the upscaled low-resolution image I'. This makes the method's local-structure prior inherit any errors in the low-resolution draft. The paper claims the module 'suppresses local distortion,' but it does not test cases where the low-resolution draft already contains distortion or where the intended high-resolution composition contains new objects absent from the draft; Fig. 15 shows a failure case but no systematic test. Please add an experiment that perturbs or distorts the low-resolution draft and reports whether the ControlNet module corrects or propagates the error, or explicitly scope the claim to drafts that are structurally sound.","section":"4.2"},{"comment":"The main quantitative claim of state-of-the-art performance rests on Table 1, but at 4x the differences from the second-best method are small (e.g., FIDc 38.10 vs. 38.36, ISc 25.59 vs. 24.62), and no standard errors, confidence intervals, or multiple-run statistics are reported. Since the method is stochastic and the datasets are the authors' own selection from LAION-5B, the reported margins may not be significant. Please report mean and variance over multiple seeds or prompts and a significance test; if the margins are not robust, temper the abstract's SOTA claim.","section":"5.2"}],"minor_comments":[{"comment":"The header 'Resolusion' should be spelled 'Resolution'.","section":"Table 1"},{"comment":"The hyperparameter c and the ControlNet step schedule are tuned on the authors' qualitative ablations (Figs. 13 and 14) and then frozen for the main benchmark; a quantitative sensitivity analysis across c values and step schedules would strengthen the claim that the chosen settings are not overfit to the displayed examples.","section":"5.1 and 5.5.2"},{"comment":"The caption uses '%' and '!' with unclear meaning; the caption text should explicitly say which symbol denotes removal and which denotes preservation.","section":"Fig. 12 caption"},{"comment":"The variable P2 is used as the number of dilated samples but is not explicitly defined; please state that P2 = hs * ws and clarify the indexing range.","section":"Eq. (14)"},{"comment":"Reference [11] lists the title as 'Democratising high-resolution image generation with no $$$'; the placeholder '$$$' should be corrected or expanded.","section":"References"},{"comment":"The GitHub URL in the abstract contains a space ('AccDiffusion v2'); the hyperlink should use 'AccDiffusion_v2' to work correctly.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The §4.3 no-op issue is the most serious problem; if the released code implements the permutation exactly as written, the 'dilated sampling with window interaction' module is inert and the contribution should be withdrawn. The other two contributions (patch-aware prompts and ControlNet structure) appear defensible in principle, but I would ask the authors to clarify the actual implementation and provide matched-seed or metric-based evidence for any interaction effect. The quantitative comparison would also benefit from error bars or significance testing before the SOTA claim can be fully supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The patch-content-aware prompt (Sec. 4.1) is a genuine and useful idea: deriving per-patch prompts from cross-attention masks of the low-resolution pass is new and the ablations support it. The ControlNet edge anchoring (Sec. 4.2) is a straightforward application of a known module, but it does work in practice and the step-schedule ablation (Fig. 14) is reasonable. Those two pieces alone would be a solid incremental contribution.\n\nThe third piece, dilated sampling with window interaction, is the problem. As written in Eqs. 16-17, a bijection is applied to the dilated samples before per-sample denoising and inverted after. The U-Net denoises each sample independently, with no cross-sample attention or mixing term. The permutation therefore commutes with the denoising, and the composition is the identity. Figure 7(a) versus 7(b) cannot demonstrate anything about the claimed interaction; any difference is initialization noise. The abstract and conclusion credit this module with suppressing repetition and distortion, but the mechanism, as specified, cannot have that effect. This is not a minor issue: the paper's central claim about its own method is unsupported for one of the three components.\n\nThe quantitative comparison is suggestive but not robust: 1,000 prompts, a single run, no seeds or variance. Several margins, like the 4x FID_c difference of 38.10 versus 38.36, are within typical noise. At 16x, the paper's own CLIP scores are below DiffuseHigh and AccDiffusion, contradicting the blanket state-of-the-art statement. To the authors' credit, the limitations section is candid about latency, dependence on the base model, invented text details, and degradation beyond 8K, which makes the rest of the evaluation more believable.\n\nThe paper deserves a serious referee, but it is not close to acceptance. The permutation module should be either removed or reworked so it actually couples the samples, and the experimental section needs matched-seed ablations and variance reporting. As is, I would not rely on the third contribution, and I would not cite the paper's full method without a major revision. The two solid modules and the honest failure analysis make it worth engaging with in review.","headline":"Two of the three modules are credible; the third, as written, is a no-op, and the paper should not be accepted until that is fixed.","tokens_in":18657,"tokens_out":1759,"would_cite":false,"duration_ms":18872,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AccDiffusion v2 claims that training-free patch-wise extrapolation can push SDXL to 2048–4096 pixel images with fewer repeated objects and less local distortion by per-patch prompts, canny-edge ControlNet conditioning, and interacting…","keywords":["high-resolution image generation","diffusion extrapolation","training-free","patch-wise denoising","patch-content-aware prompts","ControlNet","dilated sampling","object repetition suppression"],"falsifier":"Generate a low-resolution draft of a single object, then run AccDiffusion v2 at 16x with a high-resolution prompt that explicitly names additional objects (e.g., draft shows one wolf, prompt says \"three wolves running\"). Because Eq. (12) drops tokens with no low-resolution attention and Eq. (13) anchors each patch to the draft's canny edges, the output should still show one wolf; an output that adds the new objects, or a quantitative comparison in which a detector counts repeated classes and finds no suppression advantage over DemoFusion, would undercut the claim.","tokens_in":17335,"feed_emoji":"🖼️","tokens_out":8237,"duration_ms":72712,"temperature":0.7,"pith_summary":"The paper tries to show that the two main failure modes of training-free higher-resolution diffusion extrapolation—repetitive objects and locally distorted structures—have identifiable causes and can be suppressed without any retraining. It claims that passing the same entire text prompt to every image patch is what produces repeated objects, while asking a patch for a global structure it cannot locally contain is what produces distortion. On that diagnosis it builds a patch-wise pipeline on SDXL that derives a dedicated prompt for each patch from cross-attention maps, feeds each patch canny-edge structure from the low-resolution draft through ControlNet, and makes global semantic information smoother by letting dilated samples interact. If correct, this gives a practical way to push a 1024-pixel pretrained diffusion model to 2048–4096 pixel images on a single consumer GPU, with quantitative results that match or beat eight training-free baselines.","feed_headline":"Training-free 4K images from SDXL with fewer repeats and distortions","feed_subtitle":"Per-patch prompts, canny edge guidance, and interactive dilated sampling suppress the classic artifacts of resolution extrapolation.","key_machinery":"The load-bearing objects are three plug-in mechanisms operating inside latent diffusion patch-wise denoising. (1) The patch-content-aware prompt: for each word token, the cross-attention map from the low-resolution generation is thresholded at its mean, opened with morphological erosion and dilation, resized, and sampled into patches; a word is included in a patch's prompt only when its responsive mask covers enough of that patch (Eq. 12), so no patch is told to draw an object that is not there. (2) ControlNet-assisted local structure: the denoised low-resolution latent is decoded, upscaled, run through the Canny edge detector, and cropped into patch conditions that are fed, together with the patch prompt, into a ControlNet-conditioned denoiser (Eq. 13), so local geometry is anchored to the draft rather than inferred from incomplete text. (3) Dilated sampling with window interaction: the high-resolution latent is subsampled into dilation grids; before each denoising step a random bijection swaps the samples at each position inside a window, and after denoising the inverse bijection restores positions (Eqs. 16–17), letting different global-context samples inform one another. These three components are what carry the claim that repetition and distortion can be removed without changing the pretrained weights.","core_discovery":"On the paper's own terms, AccDiffusion v2 establishes that accurate higher-resolution diffusion extrapolation can be decomposed into three corrections. First, repetitive generation is the adversarial outcome of one image-level prompt applied to every patch, so the method recovers a patch-content-aware prompt for each patch by thresholding the low-resolution cross-attention map (Eq. 12), keeping only the word tokens whose attended regions actually cover that patch. Second, local distortion persists even with per-patch prompts because a prompt such as \"astronaut\" urges a patch that contains only a hand to complete the whole object; the remedy is to condition each patch's denoising on the canny edges of the upscaled low-resolution image through ControlNet (Eq. 13), so local structure no longer has to be guessed from text. Third, global semantic information suppresses both artifacts, but conventional dilated sampling denoises its samples independently, producing inconsistent global guidance; AccDiffusion v2 permutes the dilated samples within each window by a position-wise bijection before each denoising step and restores positions afterward (Eqs. 16–17), yielding smoother global context. The paper reports that on SDXL this combination reaches the best FID, IS, and CLIP scores among the compared training-free extrapolation methods at 4x, 9x, and 16x resolutions, and that the visual artifacts of repetition and distortion are visibly reduced.","pith_inferences":["An implication the paper leaves implicit: AccDiffusion v2 inherits the composition of its low-resolution draft. Because both prompt selection and canny anchoring read from that draft, a user who wants new objects or a rearranged scene at high resolution should regenerate the draft; changing only the high-resolution prompt will not introduce content the draft lacks.","The mask machinery behind Eq. (12) could be reused outside extrapolation as a cheap, model-agnostic localizer: thresholding cross-attention maps at their mean and applying morphological opening yields per-word region masks that could drive spatially targeted prompt editing, region-aware inpainting, or attention debugging.","The window-interaction permutation in Eqs. (16)–(17) is plausibly a general cure for tiled or patch-wise generative inconsistency, not just for dilation grids: any process that denoises interleaved pieces independently may suffer the same non-smooth global semantics, and the bijection trick transfers directly.","Because the distortion remedy is coupled to the canny condition, a testable extension is applying AccDiffusion v2 to other structural priors (depth, pose, or an LLM-refined per-patch prompt) to see whether the gains come from structure conditioning per se or from canny edges specifically; the paper names this direction when it suggests vision-language models for prompt refinement."],"forward_implications":["In the paper's settings, SDXL extrapolates to 2048×2048, 3072×3072, and 4096×4096 (4x, 9x, and 16x) in about 4, 15, and 35 minutes on a single consumer GPU, with the best FID/IS/CLIP scores among the compared training-free methods.","The recipe transfers to other latent diffusion backbones: the paper shows 4x generation with Stable Diffusion 1.5 (512² to 1024²) and Stable Diffusion 2.1 (768² to 1536²), with output quality tracking the backbone's quality.","Reducing the number of ControlNet steps trades local fidelity for speed; the ablation shows local artifacts reappear as ControlNet is used less often.","The three modules are complementary: removing any one of them reintroduces either small-object repetition, semantic inconsistency, or local distortion, so the full combination is what suppresses both artifact classes.","The paper's stated limits carry over: inference latency is high, quality depends on the base model, spurious text-like details can appear, and beyond about 8K (64x) detail degrades."],"supporting_citations":[{"why":"Supplies the pretrained SDXL weights and defines the 1024² training resolution that AccDiffusion v2 extrapolates from.","marker":"[7]"},{"why":"An image-wise extrapolation baseline whose memory growth motivates the patch-wise approach; used in the quantitative comparison.","marker":"[8]"},{"why":"A tuning-free image-wise extrapolation baseline compared at 4x, 9x, and 16x resolutions.","marker":"[9]"},{"why":"Source of the patch-wise denoising scheme with shifted windows and averaging; the baseline that exhibits large-object repetition.","marker":"[10]"},{"why":"Closest prior work: progressive upscaling, residual connections, and dilated sampling that AccDiffusion v2 builds on and modifies with window interaction.","marker":"[11]"},{"why":"ControlNet is the mechanism that injects canny-edge local structure into patch denoising, the paper's main new module for local distortion.","marker":"[23]"},{"why":"The conference version that first proposed patch-content-aware prompts and dilated sampling with interaction, which this journal version extends.","marker":"[24]"},{"why":"Canny edge detector produces the local-structure maps that anchor each high-resolution patch to the low-resolution draft.","marker":"[34]"},{"why":"Establishes cross-attention maps as indicators of which image regions a word token attends to, the basis for patch-content-aware prompt selection.","marker":"[38]"}],"fun_headline_variants":["Training-free high-res diffusion: per-patch prompts fix repeats and distortion","AccDiffusion v2: higher-res images without repetition or distortion","No training needed: per-patch prompts and edges fix high-res diffusion artifacts","Fix high-res diffusion glitches with per-patch prompts and edge control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The low-resolution generation is already the right content blueprint: Eq. (12) assumes its cross-attention maps tell which prompt words belong in each high-resolution patch, and Eq. (13) assumes its upscaled canny edges give the correct local structure, so if the ideal high-resolution image contains objects or composition absent from the low-resolution draft, the method is pulled back toward the draft and cannot deviate from it.","fun_headline_variants_meta":{"raw":{"variants":["Training-free high-res diffusion: per-patch prompts fix repeats and distortion","AccDiffusion v2: higher-res images without repetition or distortion","No training needed: per-patch prompts and edges fix high-res diffusion artifacts","Fix high-res diffusion glitches with per-patch prompts and edge control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000457,"raw_usage":{"total_tokens":2386,"prompt_tokens":1128,"completion_tokens":1258,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":744,"completion_tokens_details":{"reasoning_tokens":1179}},"tokens_in":744,"tokens_out":1258,"duration_ms":8690,"temperature":1.0,"reasoning_tokens":1179,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:51:51.907668+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a low-resolution draft of a single object, then run AccDiffusion v2 at 16x with a high-resolution prompt that explicitly names additional objects (e.g., draft shows one wolf, prompt says \"three wolves running\"). Because Eq. (12) drops tokens with no low-resolution attention and Eq. (13) anchors each patch to the draft's canny edges, the output should still show one wolf; an output that adds the new objects, or a quantitative comparison in which a detector counts repeated classes and finds no suppression advantage over DemoFusion, would undercut the claim.","supporting_citations":[{"cited_title":"Sdxl: Improving latent diffusion models for high-resolution image synthesis,","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained SDXL weights and defines the 1024² training resolution that AccDiffusion v2 extrapolates from."},{"cited_title":"Training-free diffusion model adaptation for variable-sized text-to-image synthesis,","cited_arxiv_id":null,"evidence_quote":"An image-wise extrapolation baseline whose memory growth motivates the patch-wise approach; used in the quantitative comparison."},{"cited_title":"Scalecrafter: Tuning-free higher- resolution visual generation with diffusion models,","cited_arxiv_id":null,"evidence_quote":"A tuning-free image-wise extrapolation baseline compared at 4x, 9x, and 16x resolutions."},{"cited_title":"Multidiffusion: Fusing diffusion paths for controlled image generation,","cited_arxiv_id":null,"evidence_quote":"Source of the patch-wise denoising scheme with shifted windows and averaging; the baseline that exhibits large-object repetition."},{"cited_title":"Demo- fusion: Democratising high-resolution image generation with no $$$,","cited_arxiv_id":null,"evidence_quote":"Closest prior work: progressive upscaling, residual connections, and dilated sampling that AccDiffusion v2 builds on and modifies with window interaction."},{"cited_title":"Adding conditional control to text-to-image diffusion models,","cited_arxiv_id":null,"evidence_quote":"ControlNet is the mechanism that injects canny-edge local structure into patch denoising, the paper's main new module for local distortion."},{"cited_title":"Accdiffusion: An accurate method for higher-resolution image generation,","cited_arxiv_id":null,"evidence_quote":"The conference version that first proposed patch-content-aware prompts and dilated sampling with interaction, which this journal version extends."},{"cited_title":"A computational approach to edge detection,","cited_arxiv_id":null,"evidence_quote":"Canny edge detector produces the local-structure maps that anchor each high-resolution patch to the low-resolution draft."},{"cited_title":"Prompt-to-prompt image editing with cross atten- tion control,","cited_arxiv_id":null,"evidence_quote":"Establishes cross-attention maps as indicators of which image regions a word token attends to, the basis for patch-content-aware prompt selection."}],"review_version":1}