{"id":"ff3072a6-e76b-4cf1-97be-b75c88f26f32","arxiv_id":"2412.03756","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A diffusion method uses coordinate noise, time-dependent Fourier attention, and a cross-attention loss to improve multi-view consistency in generated images.","lead":"This paper adds three tools to make text-to-image models generate consistent views of a scene: coordinate-based noise, Fourier attention, and a cross-attention loss. It reports better consistency than prior methods on panoramic and depth-conditioned benchmarks, though some image quality scores are mixed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed non-overlapping-region consistency gain from FBA is not directly measured; the reported overlap-PSNR and full-image Intra-LPIPS metrics cannot separate it from global or shared-noise effects.","rationale":"I read the paper as a genuine engineering contribution with unusually complete ablations and clear writing; the coordinate-noise initialization and cross-attention loss are plausible and the consistency gains over MVDiffusion, if reproducible, are meaningful. However, the central claim is specifically about non-overlapping regions, and the quantitative evaluation never isolates those regions. The reader's weakest assumption, that the frequency-filtered coordinate-noise features in Eq. 14 carry the intended scene structure, is a valid mechanism-level worry. My concern is upstream: even if that mechanism works as intended, the reported overlap-PSNR and whole-image Intra-LPIPS metrics would not reveal whether FBA is responsible for the non-overlap improvement. The ablation evidence actually suggests most of the measurable consistency gain comes from coordinate noise plus the cross-attention loss, with FBA adding little on the global metrics. This does not refute the paper, but it means the headline attribution is under-supported by the current evaluation. I would keep the reader's CONDITIONAL verdict and require a region-masked consistency analysis or a direct non-overlap metric, plus error bars, before accepting the FBA-specific claim. My agreement with the reader is partial because we identify related but distinct weaknesses: the reader questions the mechanism, whereas I question whether the evaluation can detect the mechanism's contribution at all.","tokens_in":12830,"tokens_out":4763,"duration_ms":47614,"concrete_test":"Compute non-overlap-restricted consistency metrics using the overlap masks M_ovr already defined in Eq. 15. Mask out overlapping pixels and compute PSNR and LPIPS between corresponding non-overlapping regions across views for Ours, MVDiffusion, and the Table 4 ablation variants (Shared Noise, Coord. Noise, FBA Blocks, Full Model). Report per-region metrics with multiple seeds and error bars. If the FBA vs. no-FBA difference in non-overlap regions is within noise, or if non-overlap consistency does not improve over Coord. Noise, the paper's headline attribution to FBA is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline contribution is improved multi-view consistency 'particularly in non-overlapping regions' via Fourier-based attention (Sec. 1, Sec. 3.3). The quantitative evidence does not isolate this claim. Table 2's 'Overlap PSNR' is by definition computed only in overlapping regions, and Table 3 uses the same protocol; Intra-LPIPS is an average over entire generated image pairs, so it does not localize errors to non-overlapping areas. The ablation in Table 4 sharpens the problem: adding FBA blocks (row 3) gives PSNR 23.9, ratio 0.64, and I-LPIPS 0.78, nearly identical to Coordinate Noise alone (24.2, 0.65, 0.78); the largest consistency gains appear only when the cross-attention loss is added (row 4: 24.7, 0.66, 0.75). The full model is also not best on FID/CLIP in Table 4, so the consistency differences are not explained by overall image quality. Thus the specific causal role of FBA in non-overlapping regions rests on qualitative Figures 4 and 5. If FBA only contributed through the shared coordinate-noise initialization, the reported metrics would look the same. This makes the central attribution underdetermined, independent of whether the Eq. 14 frequency-masking mechanism is semantically meaningful.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"arXiv:2412.03756 addresses multi-view text-to-image consistency in latent diffusion models with three proposed components. A coordinate-noise initialization (Eqs. 7-9) replaces pure Gaussian latents with a per-view blend of normalized coordinates or depth maps, shared Gaussian noise, and independent noise. Fourier-based attention blocks (FBA; Eqs. 10-16) attend in overlapping regions via MVDiffusion-style correspondence-aware attention and in non-overlapping regions to features filtered by a time-dependent frequency mask. A prompt cross-attention loss (Eq. 17) matches noisy-latent cross-attention maps to clean-latent maps. Experiments on Matterport3D panoramic generation and ScanNet depth-to-image generation report improved overlap PSNR ratio and intra-LPIPS relative to MVDiffusion, with mixed FID/CLIP results (better in panoramas, slightly worse in depth-to-image). The paper includes ablations of the noise weight and frequency-mask schedule in the supplement.","tokens_in":13107,"tokens_out":12577,"duration_ms":111834,"significance":"If the reported gains are reproducible, the paper is a solid engineering contribution: the method is modular, trains only added blocks while freezing a pretrained U-Net, and the supplement contains careful ablations of several design choices. The quantitative improvements over MVDiffusion on overlap PSNR ratio and intra-LPIPS are plausible and appear consistent across two datasets. The main weaknesses are that the headlined non-overlapping-region mechanism is not evaluated directly and that no uncertainty estimates are provided; both are required to support the SOTA claim.","major_comments":[{"comment":"The central claim that FBA improves multi-view consistency 'particularly in non-overlapping regions' (Sec. 1, Sec. 3.3) is not supported by the quantitative evidence. The reported Overlap PSNR ratio is, by construction, computed only in overlapping regions, and Intra-LPIPS is an average over all generated view pairs without spatial localization. In the ablation of Table 4, adding FBA Blocks (row 3: PSNR 23.9, ratio 0.64, I-LPIPS 0.78) gives essentially the same consistency numbers as Coordinate Noise alone (row 2: 24.2, 0.65, 0.78); the largest improvement appears only after the cross-attention loss is added in the Full Model (row 4: 24.7, 0.66, 0.75). Thus the specific causal role of FBA in non-overlapping regions rests on the qualitative Figures 4-5, and the measured gains could plausibly arise from the shared-noise initialization alone. Please add a non-overlap-specific metric (e.g., PSNR or LPIPS restricted to masked non-overlap regions) and an ablation that separates FBA from the cross-attention loss.","section":"Sec. 3.3 and Table 4"},{"comment":"All comparisons are reported as single point estimates with no error bars, number of independent runs, or statistical tests. Several headline differences are modest (e.g., overlap PSNR ratio 0.66 vs. 0.60 in Table 2; I-LPIPS 0.75 vs. 0.79; Table 3 ratio 0.94 vs. 0.87), and the ablation margins in Table 4 are even smaller. Because multi-view generation is stochastic and only 8 views are generated per scene, these margins may be within run-to-run variation. Please report at least three independent sampling runs with mean and standard deviation (or confidence intervals) for the main metrics, and ideally for the ablations.","section":"Sec. 4.3, Tables 1-4"},{"comment":"The values w=0.5 and the HPF-rt schedule used for the main results are selected by ablations on the same Matterport3D evaluation protocol used for the headline numbers (Tables S2 and S4). This is a form of test-set selection and can inflate the reported improvements over baselines. Please either reserve a validation split for hyperparameter selection, or explicitly report the selection protocol and quantify the sensitivity of the main comparisons to w and rt.","section":"Appendix C, Tables S2 and S4"},{"comment":"The time-dependent mask defined by rt = 1 - t/T and Mrt_F = 1 - 1_{central} has a degenerate behavior at the final denoising steps. At t=T (noisiest), rt=0 and M is all ones, so all frequencies pass; at t=1 (cleanest), rt≈1 and M is all zeros, so the frequency-filtered feature term F^{-1}(M⊙F(G)) in Eq. 14 vanishes, leaving only the positional-encoding term γ(1−rt). This is in tension with the stated design of attending to progressively higher spatial frequencies across denoising time steps and makes the attention target at the final step essentially position-only. Please clarify whether FBA is intentionally disabled near t=1, and if not, define rt so that the pass band does not collapse (e.g., rt=(t-1)/(T-1)).","section":"Eqs. 12-14"}],"minor_comments":[{"comment":"The positional encoding γ(·) is used for both the displacement in Eq. 11 and the radius in Eq. 14, but it is never defined. Please specify the encoding (e.g., sinusoidal positional encoding with which frequencies) and note whether the two uses share weights.","section":"Sec. 3.3, Eqs. 11 and 14"},{"comment":"Please clarify what Ml_t and Ml_0 represent exactly: normalized cross-attention distributions per prompt token, and which layers (and heads, if any) are aggregated. Also state the norm used in Eq. 17.","section":"Sec. 3.4, Eq. 17"},{"comment":"The conclusion states that the method 'outperform[s] SOTA baselines' and 'achieve[s] multi-view consistency while maintaining the diversity,' but Table 3 shows a worse FID than MVDiffusion (27.0 vs. 23.7). The main text qualifies this in Sec. 4.3.2, but the conclusion should be revised to avoid overstatement.","section":"Sec. 4.3.2 and Sec. 5"},{"comment":"For SyncDiffusion, the table shows '-' for overlap PSNR; please explain explicitly that the comparison is computed after splitting the single generated panorama into non-overlapping views, making overlap PSNR undefined.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"This is a respectable paper within its scope, but the gap between the advertised FBA mechanism and the measurable evidence is the main risk. I would encourage the editor to request the non-overlap-specific evaluation and error bars; without these, the central contribution is hard to distinguish from the ablated baseline. I do not suspect any author misconduct; the concern is purely about experimental support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this one with the stress test in hand. The paper is a three-part recipe: coordinate-based noise initialization, a Fourier attention block (FBA) for non-overlapping regions, and a cross-attention loss. On Matterport3D panoramas and ScanNet depth-to-image, it beats MVDiffusion on overlap PSNR ratio and intra-LPIPS, and the ablations are honest enough to show that the full model is not the best on FID/CLIP.\n\nWhat's actually new: the time-dependent high-pass frequency filtering inside attention for non-overlapping regions is not in prior work, and the coordinate noise idea is a clean way to inject pose/layout information without inversion. The writing is clear, the supplementary material is thorough, and the comparison to MVDiffusion is the right baseline for this subfield.\n\nThe soft spots are real and align with the stress test. Table 4 is the problem: adding FBA to coordinate noise barely moves any metric (PSNR 24.2 to 23.9, ratio 0.65 to 0.64, I-LPIPS unchanged). The big gains appear only after the cross-attention loss is added. The paper's headline claim about non-overlapping regions is supported only by qualitative figures, not by the quantitative metrics, which are overlap-PSNR and whole-image I-LPIPS. The authors selected w=0.5, the mask schedule, and lambda on the same benchmark, though Table S2 suggests results are not hypersensitive to w. No error bars, no code, no statistical tests. That limits confidence in the magnitude of the reported differences.\n\nNone of this kills the paper. The central idea is plausible, and the empirical work is generally careful. What it needs is a revision that either adds a metric localizing errors to non-overlapping regions or an ablation that shows FBA's contribution independently of the other two tricks. I'd like code before citing the specific numbers.\n\nSend it to review. It's a useful incremental contribution, and a sharp referee can push on the attribution question. I'd bring it to reading group as a case study in how ablations can undermine a paper's central narrative.","headline":"Solid combination of known ideas for multi-view consistency, but the paper's own ablations don't isolate the Fourier attention block as the source of its gains.","tokens_in":13639,"tokens_out":2066,"would_cite":true,"duration_ms":19363,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that multi-view diffusion images become more consistent when noise is initialized with shared coordinate structure and attention is filtered by spatial frequency, especially in non-overlapping regions.","keywords":["multi-view consistency","text-to-image diffusion","coordinate noise initialization","Fourier-based attention","panoramic image generation","depth-to-image generation","cross-attention loss","latent diffusion model"],"falsifier":"Run the panorama experiment with the Fourier attention mask replaced by a fixed random mask with the same size and total energy while keeping coordinate noise and the cross-attention loss; if the overlap-PSNR ratio and intra-LPIPS stay near 0.66 and 0.75, the frequency-selection mechanism is not carrying the claimed alignment. Alternatively, measure whether the attention weights in non-overlapping regions correlate with ground-truth scene layout when ground truth is available; if they do not, the Fourier-based attention blocks are not actually aligning structure.","tokens_in":12631,"feed_emoji":"🖼️","tokens_out":9293,"duration_ms":83312,"temperature":0.7,"pith_summary":"Diffusion models can now draw a single image from text, but drawing several views of the same scene usually fails: things in one view do not match things in another, especially in areas where the views do not overlap. This paper claims that the failure can be fixed by changing how generation starts and what the attention layers look at. It initializes each view's noise with shared random noise blended with low-frequency coordinate and depth structure, adds an attention block that in non-overlapping regions attends to progressively higher spatial frequencies as denoising proceeds, and adds a loss that pins prompt-to-image attention to ground-truth layout. If the claim holds, panoramic and depth-conditioned generation becomes visibly more coherent, with textures and objects staying put across large view changes; the paper reports overlap-PSNR ratio rising from 0.60 to 0.66 in panoramas and from 0.87 to 0.94 in depth-to-image. The recipe works on top of an existing latent diffusion model without retraining its backbone, so the fix is portable.","feed_headline":"Fourier attention keeps multi-view diffusion images consistent","feed_subtitle":"Shared coordinate noise and a time-dependent frequency mask cut texture drift in panoramas and depth-to-image generation.","key_machinery":"The load-bearing object is the coordinate-noise initialization of Eq. 8--9: a per-view linear blend $w c^i + (1-w)\\epsilon_{\\mathrm{shared}}$ of shared Gaussian noise with low-frequency scene information (normalized depth maps, or pixel coordinates warped into the center view). The Fourier-based attention (FBA) block is the mechanism that exploits it: it takes U-Net features produced from coordinate noise, applies a 2D FFT, multiplies by the high-pass mask $M_{r_t}^F$ with radius $r_t = 1 - t/T$, applies the inverse FFT plus a positional encoding, and merges these filtered non-overlap features with correspondence-aware features from the overlapping-region mask before softmax attention. The prompt cross-attention loss $L_{\\mathrm{XA}}^l = \\|M_t^l - M_0^l\\|$ anchors prompt-to-view attention to clean-latent attention maps. The FFT mask does the central work: it turns the low-frequency bias of the initialization into a time-dependent curriculum from broad structure to fine detail.","core_discovery":"The paper's central claim is that multi-view consistency improves when noise initialization and attention are both made aware of the scene's spatial structure. Instead of sampling independent Gaussian noise per view, the method uses coordinate noise: per-view latents built as $\\hat{z}_T^i = \\sqrt{\\bar{\\alpha}_T}\\hat{\\epsilon}^i + \\sqrt{1-\\bar{\\alpha}_T}\\epsilon^i$ with $\\hat{\\epsilon}^i = w c^i + (1-w)\\epsilon_{\\mathrm{shared}}$, where $c^i$ is normalized depth or pixel coordinates transformed into a reference view. The Fourier-based attention block then gathers U-Net features generated from this coordinate noise, applies an FFT, keeps only frequencies above a time-dependent radius ($r_t = 1-t/T$), and combines those filtered features with correspondence-aware features from overlapping regions before softmax attention. A prompt cross-attention loss, $L_{\\mathrm{XA}}^l = \\|M_t^l - M_0^l\\|$, anchors each view's prompt-to-image attention maps to the noise-free ground-truth maps. On the paper's measurements this yields overlap-PSNR ratios of 0.66 in panoramas and 0.94 in depth-to-image, up from 0.60 and 0.87 for the MVDiffusion baseline, and intra-LPIPS decreases from 0.80 to 0.75 in panoramas.","pith_inferences":["Inference: the time-dependent frequency schedule suggests a testable extension in which the radius $r_t$ is matched to the noise level $\\bar{\\alpha}_t$ rather than a linear schedule; the paper does not test this variant.","Inference: coordinate noise requires only camera pose and optional depth, so the same initialization could be dropped into video-diffusion and 3D generation pipelines to stabilize global appearance without retraining the backbone.","Inference: the cross-attention loss needs ground-truth multi-view images to anchor the attention maps, which limits it to datasets with posed multi-view captures; in unposed settings, a self-supervised anchor such as prediction agreement across views would be a natural replacement.","Inference: the reported depth-to-image FID is slightly worse than the main baseline, which the authors attribute to imprecise per-view prompts; a testable implication is that with better cross-view-consistent captions, the FID gap would close or reverse."],"forward_implications":["Panorama generation becomes more globally coherent: the overlapping-PSNR ratio rises from 0.60 to 0.66 and intra-LPIPS falls from 0.80 to 0.75, meaning colors and textures match better across the full sweep.","Depth-conditioned multi-view generation improves from a 0.87 to a 0.94 overlap-PSNR ratio, so small objects, large objects, and room-scale environment textures change less from view to view.","Prompt fidelity improves: scenes generated from prompts like \"a house with a pool in the backyard\" actually contain both the house and the pool, because the cross-attention loss pins prompt-to-image attention to ground-truth layout.","The three components are additive: shared noise alone helps, coordinate noise helps more, adding the Fourier-based attention blocks helps further, and the full model with the cross-attention loss gives the best numbers in the paper's ablation.","Because the U-Net backbone is frozen and only new attention blocks are trained, the recipe can be layered onto an existing single-view latent diffusion model."],"supporting_citations":[{"why":"Supplies the correspondence-aware attention (CAA) modules and the evaluation protocol that the Fourier-based attention extends and compares against.","marker":"[24]"},{"why":"Provides the latent diffusion model architecture and the single-view baseline on which all methods are built.","marker":"[21]"},{"why":"Establishes the training/inference signal-to-noise gap that motivates injecting shared low-frequency structure into the noise initialization.","marker":"[13]"},{"why":"Shows that low spatial frequency information in initialization improves consistency; the coordinate noise builds on this idea without requiring real images.","marker":"[27]"},{"why":"Demonstrates cross-attention and low-frequency conditioning for video generation and serves as a comparison point for the non-overlap limitation.","marker":"[20]"},{"why":"Introduces shared versus independent noise priors for video diffusion, grounding the shared-noise component of Eq. 7.","marker":"[6]"},{"why":"Provides the cross-attention loss formulation that the paper extends to the multi-view setting.","marker":"[16]"},{"why":"Serves as a panorama-generation baseline whose synchronized diffusion output is compared on FID, CLIP Score, and Intra-LPIPS.","marker":"[12]"},{"why":"Serves as a depth-conditioning baseline in the depth-to-image experiments.","marker":"[30]"}],"fun_headline_variants":["Coordinate noise and Fourier attention align multi-view images","Coordinate noise and Fourier attention boost multi-view consistency","Multi-view diffusion gets consistent with coordinate noise and Fourier attention","Fourier attention and coordinate noise tie views together","Coordinate noise plus Fourier attention keep views aligned"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the premise that blending shared random noise with low-frequency coordinate and depth maps actually plants consistent scene layout into the model's internal representations, so that the frequency-filtered attention aligns real structure rather than a shared random signal.","fun_headline_variants_meta":{"raw":{"variants":["Coordinate noise and Fourier attention align multi-view images","Coordinate noise and Fourier attention boost multi-view consistency","Multi-view diffusion gets consistent with coordinate noise and Fourier attention","Fourier attention and coordinate noise tie views together","Coordinate noise plus Fourier attention keep views aligned"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000691,"raw_usage":{"total_tokens":3145,"prompt_tokens":980,"completion_tokens":2165,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":2092}},"tokens_in":596,"tokens_out":2165,"duration_ms":14061,"temperature":1.0,"reasoning_tokens":2092,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:06:41.613445+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the panorama experiment with the Fourier attention mask replaced by a fixed random mask with the same size and total energy while keeping coordinate noise and the cross-attention loss; if the overlap-PSNR ratio and intra-LPIPS stay near 0.66 and 0.75, the frequency-selection mechanism is not carrying the claimed alignment. Alternatively, measure whether the attention weights in non-overlapping regions correlate with ground-truth scene layout when ground truth is available; if they do not, the Fourier-based attention blocks are not actually aligning structure.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the latent diffusion model architecture and the single-view baseline on which all methods are built."},{"cited_title":"Common diffusion noise schedules and sample steps are flawed","cited_arxiv_id":null,"evidence_quote":"Establishes the training/inference signal-to-noise gap that motivates injecting shared low-frequency structure into the noise initialization."},{"cited_title":"Preserve your own correlation: A noise prior for video diffusion models","cited_arxiv_id":null,"evidence_quote":"Introduces shared versus independent noise priors for video diffusion, grounding the shared-noise component of Eq. 7."},{"cited_title":"Zero-shot image-to-image translation","cited_arxiv_id":null,"evidence_quote":"Provides the cross-attention loss formulation that the paper extends to the multi-view setting."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Serves as a depth-conditioning baseline in the depth-to-image experiments."}],"review_version":1}