{"id":"703c6501-c07d-47ae-bc36-a2a34bfdaf66","arxiv_id":"2412.16645","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A neural network that fuses NIR and RGB images in the frequency domain to denoise RGB images, achieving state-of-the-art PSNR on DVD and IVRG benchmarks.","lead":"This paper presents a neural network that uses near-infrared (NIR) images to help clean noisy RGB photos, by exploiting a frequency-domain pattern where the two image types complement each other. The claimed result is better denoising quality than existing methods on standard benchmarks, at moderate computational cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FEFM equations (4)-(5) apply Softmax and feature mixing to complex-valued frequency-domain tensors without specifying any real-valued reduction, leaving the core fusion module underspecified and not reproducible from the paper text alone.","rationale":"The reader's weakest assumption concerns the statistical reliability of the 30-pair frequency-correlation prior. That is a legitimate concern, but it is about the motivating analysis rather than the core method itself: even if the prior is imperfect, the FDSM and FEFM could still learn useful behavior from data. The more fundamental issue is that the FEFM equations are not internally consistent as written. FQ, FK, and FCFR are complex-valued, yet the paper applies operations (Softmax, multiplication, subtraction) that are only defined for real tensors, with no stated conversion back to real feature maps. This is not a stylistic quibble: it is a mathematical gap in the description of a key component. The central claim of state-of-the-art performance depends on FEFM, so if the equations cannot be implemented as written, the claim cannot be verified without additional specification or code. This concern is different from the reader's, though related to the overall reproducibility issue raised by the absence of code at review time. I do not think it warrants rejection, because the empirical results may well be correct and the authors may have used a standard real-valued approximation; the issue is that the paper as written is incomplete. The reader's verdict of CONDITIONAL remains appropriate, with the condition being that the authors clarify the complex-to-real mapping and release code. Therefore, I recommend no change to the verdict, but I flag this specific technical gap as the most load-bearing point of scrutiny.","tokens_in":13911,"tokens_out":8440,"duration_ms":75737,"concrete_test":"Implement Eq. (4) literally with complex tensors in PyTorch (e.g., torch.fft.fft2 on Q and K, then compute FQ @ FK.transpose(-2,-1) / alpha and call softmax). The operation should fail or be mathematically undefined for complex inputs. Then implement the most natural correction (e.g., using the real part or magnitude before softmax and after the inverse FFT in Eq. (5)), retrain on the DVD dataset under the exact protocol in Section 4.1, and compare the reported σ=2 PSNR of 32.43 dB in Table 1. If the corrected implementation does not reproduce that number, the published equations do not describe the evaluated model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central architecture relies on the Frequency Exhaustive Fusion Mechanism (FEFM), but its defining equations are not mathematically well-defined as written. In Eq. (4), FQ and FK are produced by applying a 2D-DFT to real feature maps, so they are complex-valued. The expression FQ · F_K^T / α is therefore complex, and applying Softmax to a complex matrix is undefined (Softmax is conventionally defined only for real vectors). Similarly, the Hadamard product FQ ⊙ FK is complex, and multiplying it by Softmax(...) does not yield a well-defined real output. In Eq. (5), FCFR is complex, so F^{-1}(FCFR) is a complex spatial-domain tensor; the operation V − λ V ⊙ F^{-1}(FCFR) would produce a complex feature map, yet the subsequent layers (convolution, residual addition) require real inputs. The paper never states that the real part is taken, that magnitudes are used, or that any other projection to real values is applied. Since CFR and DFR are the two components of FEFM, and FEFM is one of the two headline contributions used to achieve the reported gains, the published description of the method is internally incomplete. Without released code—the abstract says code 'will be released'—a reader cannot determine how these complex operations are implemented in practice, making the central claim neither reproducible nor fully specified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FCENet, a two-stage U-Net architecture for NIR-assisted RGB image denoising. The first stage pre-denoises the RGB image; the second stage fuses NIR and RGB features using two newly proposed modules: Frequency Dynamic Selection Mechanism (FDSM) and Frequency Exhaustive Fusion Mechanism (FEFM). The design is motivated by a claimed frequency correlation prior: across the same scene, noisy RGB becomes less similar to clean RGB at higher frequencies, while NIR becomes more similar to clean RGB at higher frequencies. The authors evaluate on the DVD and IVRG datasets under several noise levels, report PSNR/SSIM improvements over existing NIR-assisted and single-image denoising methods, and provide ablations showing that FDSM and FEFM each contribute to the final performance.","tokens_in":14180,"tokens_out":5553,"duration_ms":47841,"significance":"If the results hold, the paper would make a useful contribution to NIR-assisted denoising by reframing cross-field fusion as frequency-domain selection and fusion. The experimental setup is a strength: all compared methods are retrained on the same training sets, the ablations in Tables 3-5 show that each proposed component contributes, and the method reports favorable FLOPs/parameter trade-offs. The frequency correlation prior is presented as a data-derived observation rather than a fitted parameter, so there is no obvious circularity in the evaluation. However, the central architectural equations for FEFM are not mathematically well-defined as written, and the prior is supported by only 30 image pairs without statistical quantification; both issues directly affect the reproducibility and credibility of the central claims.","major_comments":[{"comment":"The core FEFM equations are underspecified: FQ and FK are 2D-DFTs of real feature maps and are therefore complex-valued, so Eq. (4) applies Softmax to a complex matrix and computes a Hadamard product of complex matrices without any stated real-valued reduction. Eq. (5) then forms V - λ V ⊙ F^{-1}(FCFR), which is complex-valued, yet the subsequent convolutional and residual operations require real feature maps. Since CFR and DFR are the two headline components of FEFM, and Table 3 attributes a 0.83 dB gain to FEFM, the published description cannot be implemented or reproduced as written. Please specify the complex-to-real mapping (e.g., taking the real part, magnitude, or a learned projection), define the dimensions of the matrix products in Eq. (4), and clarify how the final fused result combines Q with the output of CFR.","section":"Sec. 3.4, Eqs. (4)-(5)"},{"comment":"The frequency correlation prior is the load-bearing motivation for both FDSM and FEFM, but it is established from only 30 randomly selected image pairs, with no error bars, confidence intervals, or statistical test. Fig. 3 shows individual curves rather than an aggregate summary, so the claimed monotonic trends are not demonstrated to hold across the full training and test distributions, different noise levels, or different sensor/scene types. Please provide a quantitative summary over the full DVD and IVRG sets (e.g., mean ± std of the SSIM-frequency curves), state which dataset the 30 pairs were drawn from, and ideally test the monotonicity claim statistically.","section":"Sec. 3.1, Fig. 3"},{"comment":"The claim that FCENet outperforms all compared methods rests on single-run PSNR/SSIM differences, some of which are small (e.g., Table 1, σ=6: FCENet-L 28.36 vs. NIR-Restormer 28.31; Table 2, σ=75: FCENet-L 25.33 vs. NIR-Restormer 25.30). Without multiple seeds, error bars, or significance tests, it is difficult to judge whether these differences are beyond run-to-run variation. Please report means and standard deviations over at least three training runs, or otherwise justify that the reported single-run differences are stable.","section":"Tables 1-2 and Sec. 4.2"}],"minor_comments":[{"comment":"The subsection heading \"Evaluations on the DVN dataset\" appears to be a typo for \"DVD dataset.\"","section":"Sec. 4.2"},{"comment":"The high-pass filter H(·) is not precisely defined; please specify whether it is a radial frequency mask, how the cutoff frequency is normalized, and how the filtered frequency-domain representation is converted back to the spatial domain.","section":"Eq. (1)"},{"comment":"In Eq. (2), G is described as containing k learnable frequency-domain filtering kernels, but it is not stated whether DFI and G are real-valued or complex-valued. Since F(I) in Eq. (3) is complex, please clarify the domain of the dynamic filter and the intended multiplication.","section":"Sec. 3.3, Eq. (2)"},{"comment":"The header of Table 3 is confusing because the first column is also labeled \"Baseline\" and row (a) has a checkmark only under that column; please rename the columns to explicitly indicate the configuration (e.g., \"FDSM\" and \"FEFM\" with a separate \"w/o modules\" column).","section":"Table 3"},{"comment":"The training noise specification says Gaussian-Poisson mixed noise with a range of 1 to 16, while Table 1 evaluates at σ=2, 4, and 6; please explain how these test noise levels relate to the training noise range and whether the model sees the exact test noise levels during training.","section":"Sec. 4.1 and Table 1"},{"comment":"The final paragraph notes that existing methods may fail on significantly misaligned cross-field images and lists this as future work; please state explicitly whether FCENet inherits this limitation and confirm that the DVD and IVRG evaluation sets do not contain significant misalignment.","section":"Sec. 5, Conclusion"},{"comment":"The notation in Fig. 5 (QF, KF, V) does not fully match the text in Sec. 3.4 (Q, K, V, FQ, FK); please align the notations for readability.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper addresses a relevant problem and the experimental setup has several strengths, including retraining all comparison methods on the same data. The main concern is that the central FEFM equations are incomplete as written, which prevents reproduction from the paper text alone; this is fixable but requires the authors to specify the complex-to-real operations and ideally release the code. The statistical support for the frequency prior should also be strengthened. I do not see evidence of problematic citation practices or misconduct; the issues are technical and presentation-level."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on arXiv:2412.16645. It's a solid, useful paper in NIR-assisted denoising, but the main method section has a real gap that needs fixing before I'd trust the architecture as described.\n\nWhat's genuinely new: the frequency correlation prior. The authors show with a simple SSIM-at-cutoff analysis that noisy RGB gets less similar to clean RGB as frequency increases, while NIR gets more similar. That's a clean empirical observation, and it directly motivates their two modules. FDSM uses input-dependent frequency filtering to select complementary features; FEFM separates common-feature reinforcement from differential-feature reinforcement. The ablations in Tables 3-5 are consistent: each component adds ~0.3-0.5 dB, and the full model beats strong baselines on both DVD and IVRG while being far cheaper than Restormer-style transformers. The light version at 1.16M parameters and 5.7 GFLOPs is a nice practical result. All comparisons are retrained on the same protocol, which is better than many papers in this area.\n\nNow the soft spots. The serious one: Eqs. (4) and (5) are not mathematically well-defined as written. FQ and FK are Fourier transforms of real feature maps, so they're complex. Softmax of a complex matrix isn't defined, and the Hadamard product followed by the inverse DFT in Eq. (5) would give a complex feature map, yet the next layers expect real values. The paper never says 'take the real part,' 'use magnitudes,' or anything else. This isn't a nitpick—it's the core fusion module. Without code (and the repo isn't live yet), a reader cannot know what the network actually computes. The authors need to spell this out or release code; probably both.\n\nThe second soft spot is the prior itself. It's measured on 30 randomly selected pairs, with no variance or significance test. That's a small sample for a claimed universal trend. The trend might hold—the learned filters in Fig. 8 are consistent with it—but the evidence is thin. For a paper whose architecture is fully built on this prior, I want to see the analysis on the full test set or at least error bars.\n\nMinor: no error bars on the PSNR gains across runs; the gains over NIR-Restormer are 0.3-0.5 dB, which could partly be run variance. And the misalignment limitation is acknowledged in the conclusion but left unsolved, which is honest but means the method only works on reasonably aligned inputs.\n\nOverall: this is a real incremental contribution with a clear empirical motivator. It deserves a serious referee. I'd send it out, but with a request that the authors fix the complex-domain specification and strengthen the prior analysis. If the equations stay as-is, the methods section isn't reproducible.","headline":"Solid NIR-assisted denoising paper with a real empirical prior, but the fusion module's equations are underspecified and need a fix before I'd trust the architecture as written.","tokens_in":14744,"tokens_out":2877,"would_cite":true,"duration_ms":23884,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frequency-domain prior for NIR-assisted denoising: keep RGB low-frequency color, take NIR high-frequency detail.","keywords":["image denoising","near-infrared imaging","frequency domain analysis","cross-field fusion","dynamic convolution","attention mechanism","guided image restoration"],"falsifier":"Compute the SSIM-versus-cutoff-frequency curves on the full DVD and IVRG test sets, or on a new NIR-RGB dataset, with per-frequency error bars. If the noisy-RGB curve does not consistently fall while the NIR curve rises, or if the ordering reverses at any band, the frequency-correlation prior is contradicted and the FDSM/FEFM design loses its motivating premise.","tokens_in":13717,"feed_emoji":"🧩","tokens_out":5275,"duration_ms":41430,"temperature":0.7,"pith_summary":"The paper establishes a frequency-correlation prior for NIR-assisted RGB denoising: in the same scene, noisy RGB images become progressively less similar to the clean image from low to high frequency, while NIR images become progressively more similar. The authors argue that this complementary relationship makes frequency-domain selection the right way to fuse the two fields. They build FCENet, which uses a Frequency Dynamic Selection Mechanism to filter useful information from each field and a Frequency Exhaustive Fusion Mechanism to combine common and differential features. On the DVD and IVRG datasets, FCENet reports higher PSNR and SSIM than prior single-image and NIR-assisted denoisers, including a light version with only 1.16 million parameters.","feed_headline":"Denoised by frequency: NIR supplies the detail, RGB the color","feed_subtitle":"FCENet reports higher PSNR on DVD and IVRG by selecting NIR high-frequency features and RGB low-frequency color.","key_machinery":"The Frequency Dynamic Selection Mechanism (FDSM) generates input-dependent frequency-domain filters by learning linear combinations of k learnable filter kernels from aggregated NIR and RGB features, then applies these filters via element-wise multiplication in the 2D-DFT domain. The Frequency Exhaustive Fusion Mechanism (FEFM) has two branches: Common Feature Reinforcement (CFR) builds a frequency-domain correlation map that combines point-wise products with long-range channel correlations, and Differential Feature Reinforcement (DFR) uses a differential cross-attention to add missing high-frequency NIR features back into the fused representation. The whole network is a two-stage U-Net with a Supervised Attention Module connecting the pre-denoised RGB stage to the NIR-fusion stage.","core_discovery":"The central discovery is the cross-field frequency correlation prior, measured by applying high-pass filters at increasing cutoff frequencies to 30 randomly selected NIR-RGB image pairs and computing SSIM between the filtered targets and the clean RGB image. The paper finds that for noisy RGB, SSIM with clean RGB falls as frequency rises, while for NIR the SSIM with clean RGB rises. The authors then claim that this prior justifies a frequency-domain fusion framework: FDSM learns input-dependent frequency filters that retain low-frequency color from RGB and mid-to-high-frequency texture from NIR, and FEFM reinforces common features through a frequency-domain correlation map while supplementing differential high-frequency features from NIR via differential cross-attention. With this design, FCENet reaches 32.43 dB PSNR on DVD at noise level σ=2 and 33.26 dB on IVRG at σ=25, outperforming all compared methods.","pith_inferences":["The paper does not claim, but it follows that the same frequency-selection structure could transfer to other guided restoration tasks, such as RGB-depth super-resolution or flash/no-flash photography, where one modality supplies high-frequency structure and the other supplies low-frequency color.","Because the prior is measured on 30 pairs with no error bars, a direct stress test would be to recompute the SSIM-vs-frequency curves separately per noise level and per scene; if the NIR curve does not consistently rise, the design's motivation weakens.","The paper admits that significantly misaligned cross-field images break existing methods; a testable extension is to insert an alignment module before FCENet and measure whether the frequency-selection gains are preserved under realistic misalignment."],"forward_implications":["If the frequency-correlation prior holds broadly, NIR-assisted denoising is better formulated as frequency-domain selection than as spatial fusion, which should reduce artifacts caused by color and structure inconsistency between fields.","Restoring high-frequency texture from NIR while preserving low-frequency color from RGB yields higher PSNR and SSIM than single-image denoisers or additive fusion, as shown on DVD and IVRG.","Dynamic frequency-domain filtering is parameter-efficient: FCENet-L with 1.16 million parameters outperforms heavier baselines.","The two-stage design, where RGB is pre-denosed before fusion, reduces the modality gap and makes subsequent frequency fusion more effective."],"supporting_citations":[{"why":"SSIM is used as the similarity measure to compute the frequency-correlation prior curves.","marker":"[40]"},{"why":"Provides the DVD dataset with NIR-RGB pairs and the DVN baseline method for comparison.","marker":"[18]"},{"why":"Introduces frequency-domain self-attention that CFR extends with both point-wise and long-range correlations.","marker":"[21]"},{"why":"Supplies the dynamic-convolution approach that FDSM adapts to generate frequency-domain filters.","marker":"[6]"},{"why":"NIR-Restormer is a state-of-the-art NIR-assisted baseline and the real-world benchmark comes from this work.","marker":"[43]"},{"why":"MNNet provides a model-driven baseline and the experimental protocol used for the IVRG dataset.","marker":"[44]"},{"why":"Charbonnier loss is the core supervised loss for both network stages.","marker":"[5]"}],"fun_headline_variants":["NIR high-freq, RGB low-freq: split for cleaner denoising","Let NIR handle texture, RGB color, to boost denoising","Frequency-wise fusion: NIR detail, RGB color","NIR adds high-freq detail, RGB retains low-freq color","Exploit NIR-RGB frequency correlation for stronger denoising"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire fusion scheme rests on the measured trend that noisy RGB similarity to clean decreases with frequency while NIR similarity increases, yet the trend is computed on only 30 image pairs with no reported variance or statistical test.","fun_headline_variants_meta":{"raw":{"variants":["NIR high-freq, RGB low-freq: split for cleaner denoising","Let NIR handle texture, RGB color, to boost denoising","Frequency-wise fusion: NIR detail, RGB color","NIR adds high-freq detail, RGB retains low-freq color","Exploit NIR-RGB frequency correlation for stronger denoising"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000702,"raw_usage":{"total_tokens":3185,"prompt_tokens":976,"completion_tokens":2209,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":2115}},"tokens_in":592,"tokens_out":2209,"duration_ms":13675,"temperature":1.0,"reasoning_tokens":2115,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:22:34.709439+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the SSIM-versus-cutoff-frequency curves on the full DVD and IVRG test sets, or on a new NIR-RGB dataset, with per-frequency error bars. If the noisy-RGB curve does not consistently fall while the NIR curve rises, or if the ordering reverses at any band, the frequency-correlation prior is contradicted and the FDSM/FEFM design loses its motivating premise.","supporting_citations":[{"cited_title":"Two deterministic half-quadratic regular- ization algorithms for computed imaging","cited_arxiv_id":null,"evidence_quote":"Charbonnier loss is the core supervised loss for both network stages."},{"cited_title":"Image quality assessment: from error visibility to structural similarity","cited_arxiv_id":null,"evidence_quote":"SSIM is used as the similarity measure to compute the frequency-correlation prior curves."},{"cited_title":"Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior","cited_arxiv_id":null,"evidence_quote":"Provides the DVD dataset with NIR-RGB pairs and the DVN baseline method for comparison."},{"cited_title":"Efficient frequency domain-based trans- formers for high-quality image deblurring","cited_arxiv_id":null,"evidence_quote":"Introduces frequency-domain self-attention that CFR extends with both point-wise and long-range correlations."},{"cited_title":"Dynamic convolution: Attention over convolution kernels","cited_arxiv_id":null,"evidence_quote":"Supplies the dynamic-convolution approach that FDSM adapts to generate frequency-domain filters."},{"cited_title":"A model-driven net- work for guided image denoising","cited_arxiv_id":null,"evidence_quote":"MNNet provides a model-driven baseline and the experimental protocol used for the IVRG dataset."}],"review_version":1}