REVIEW 3 major objections 5 minor 10 references
A Comparative Study of NAFNet Baselines for Image Restoration
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read On CIFAR10 restoration, NAFNet's SimpleGate and linear attention outperform GELU, ECA, and no-attention baselines.
desk verdict A small honest NAFNet ablation whose own Table I undermines the abstract's LayerNorm claim; worth refereeing but needs multi-seed data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the NAFNet residual block, which replaces explicit nonlinear activations with SimpleGate—splitting the feature map along the channel dimension and multiplying the two halves element-wise—and replaces sigmoid-based channel attention with a purely linear Simplified Channel Attention computed from global average pooled features. The block also contains LayerNorm, a $1\times1$ convolution, a $3\times3$ depthwise convolution, and a residual skip connection. The ablation isolates each design decision by swapping that single component (GELU for SimpleGate, ECA for SCA, GroupNorm for LayerNorm, or removal of attention) while keeping the rest of the architecture fixed.
What would settle it
Run each of the five variants (baseline, A1, A2, A3, A4) multiple times with different random seeds under identical settings and compare the spread of PSNR, SSIM, and LPIPS across runs. If the per-variant ranges overlap the gaps between variants—for example, if the baseline and A1 PSNR ranges overlap—then the claimed ranking is not established by the current numbers.
Extended reading notes
Core claim
The paper's central claim is that NAFNet's core components generalize to a small-scale benchmark: SimpleGate gating outperforms GELU activation, the nonlinearity-free Simplified Channel Attention (SCA) outperforms ECA's sigmoid-based attention, and attention as a whole is indispensable even when other components are simplified. The reported numbers place the baseline at 29.37 dB PSNR, 0.9565 SSIM, and 0.0073 LPIPS, with the GELU variant at 29.14 dB, the ECA variant at 28.86 dB, and the no-attention variant at 28.80 dB; the GroupNorm variant reaches 29.38 dB and best PSNR. The authors read these results as validating NAFNet's simplified design while noting that GroupNorm is a viable normalization at low resolution.
Load-bearing premise
The conclusions depend on the premise that a single training run per variant, with no reported random seeds or error bars, yields differences that are reliable enough to rank the design choices; if run-to-run variance is comparable to the observed margins, the ordering of SimpleGate over GELU and GroupNorm over LayerNorm loses support.
Editorial extensions
If this is right
- SimpleGate can substitute for GELU while preserving restoration quality, so nonlinear activation functions are not necessary for this task.
- Channel attention is load-bearing: removing it costs texture and fine detail, so any further simplified block should retain some global feature modulation.
- GroupNorm matches or beats LayerNorm on 32x32 images, suggesting normalization choice should be tuned per resolution.
- NAFNet's design transfers to small datasets, so the architecture is not dependent on large benchmarks for its advantage.
Reading between the lines
- A multi-seed repetition of these experiments, not reported in the paper, could settle whether the sub-0.2 dB gaps between baseline, A1, and A3 are real; if the per-variant variance overlaps those gaps, only the attention-removal result would remain robust.
- Because LPIPS and PSNR order the variants differently, retraining with a perceptual or adversarial loss could plausibly change which design wins, which the authors list as future work.
- The GroupNorm near-tie suggests that a systematic search over group counts and normalization placements, which the paper does not run, could yield further gains on low-resolution restoration.
- Testing the same four ablated variants on higher-resolution benchmarks like GoPro or SIDD would tell whether the small-scale conclusions about SimpleGate and SCA are resolution-dependent.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an empirical ablation study of NAFNet components on a small-scale image restoration task. The authors build a CIFAR10 benchmark with synthetic Gaussian blur and noise, implement a NAFNet-style baseline using SimpleGate, Simplified Channel Attention (SCA), and LayerNorm, and compare it against four variants: GELU instead of SimpleGate (A1), ECA instead of SCA (A2), GroupNorm instead of LayerNorm (A3), and removal of attention entirely (A4). They report PSNR, SSIM, and LPIPS, provide qualitative example outputs, and conclude that the NAFNet design generalizes to small images, with SimpleGate and simplified attention being effective and LayerNorm important for stable training.
Significance. If the conclusions were fully supported, the paper would provide a useful independent, small-scale validation of NAFNet's design principles and a lightweight testbed for future ablations. The authors genuinely compare alternative components rather than deriving conclusions from the original NAFNet paper, and the inclusion of LPIPS in addition to PSNR/SSIM is a positive feature. However, the current experimental evidence is not strong enough to carry the paper's claims: the reported effects are small, all comparisons are based on single training runs without reported seeds or error bars, and the headline claim about LayerNorm is contradicted by the paper's own table. The significance of the study is therefore conditional on additional experimental rigor and a revision of the central claims.
major comments (3)
- [Abstract, Section IV Table I, Section V, Section VI] The paper's central claim that 'LayerNorm proves to be important for stable training' is contradicted by its own Table I. The baseline (LayerNorm) achieves PSNR 29.37, SSIM 0.9565, LPIPS 0.0073, while A3 (GroupNorm) achieves PSNR 29.38, SSIM 0.9562, LPIPS 0.0079. The Discussion and Conclusion explicitly state that GroupNorm 'slightly outperformed' the baseline and 'can be a viable and even superior' normalization choice. In addition, no training-stability evidence is provided anywhere in the manuscript: there are no training curves, convergence diagnostics, or gradient statistics. The abstract and conclusion should be revised to state that LayerNorm and GroupNorm perform comparably, and the 'stable training' claim should be removed or supported with actual stability measurements.
- [Section III.D, Section IV Table I] All variant rankings rest on a single training run per model, with no random seeds, no error bars, and no significance tests. The observed margins are small: A1 versus baseline is 0.23 dB, A2 versus baseline is 0.51 dB, A4 versus baseline is 0.57 dB, and A3 versus baseline is 0.01 dB. For a 32x32 CIFAR10 task with batch size 16 and 50 epochs, run-to-run variation can plausibly exceed these margins. The paper should report the random seed(s), run each variant multiple times (at least 3-5 seeds), report mean and standard deviation, and, where possible, provide pairwise significance tests or a paired bootstrap. Without this, the claimed advantage of SimpleGate over GELU and SCA over ECA is not statistically established.
- [Section I.D, Section III.A, Section III.D] The manuscript does not provide enough information to reproduce the experiments. The code statement says only that the source code is 'available on GitHub at this repository' without a URL, and the architecture description omits the exact number of NAF blocks, channel widths at each stage, the learning rate schedule, the early stopping threshold, and any data augmentation. Since the contribution is an empirical comparison, these missing details are load-bearing for reproducibility. Please provide the repository URL and a complete specification of hyperparameters and architecture dimensions, either in the paper or in a supplementary document.
minor comments (5)
- [Section II.C] There is a typo: 'intoduces' should be 'introduces'. Also, the capitalization of NAFNet is inconsistent (e.g., 'NAFNET' appears in Section II.C).
- [Section IV, Figure 3] Figure 3 shows output panels for the baseline, A1, A2, and A3, but not for A4, even though Section IV states that removing attention notably hurts texture recovery and fine detail. Please include an A4 output panel or clarify that the qualitative claim is based on unshown results.
- [Section III.E] The LPIPS metric is listed, but the specific pretrained network (e.g., VGG or AlexNet) and the version used are not specified. This should be stated for reproducibility.
- [Section IV] The phrase 'The A3 model showed pretty much similar results to the baseline' is informal; consider rewriting as 'The A3 model produced results very similar to the baseline.'
- [Section VI] The conclusion says 'SimpleGate slightly outperformed GELU' and 'SCA proved more effective than ECA'; given the lack of error bars, 'proved' and 'outperformed' are too strong. Recommend phrasing such as 'tended to perform better in this single-run comparison.'
Circularity Check
No circularity: the ablation compares independently trained variants against an externally cited baseline; all reported metrics are measured outputs, not quantities defined from the inputs.
full rationale
The paper makes no derivation or first-principles prediction; it is a controlled empirical ablation. The baseline architecture is imported from Chen et al. [1] by explicit citation, but every claim about component importance is tested by independent training runs: A1 replaces SimpleGate with GELU, A2 replaces SCA with ECA, A3 replaces LayerNorm with GroupNorm, and A4 removes attention. These comparisons do not reduce by construction: the metrics (PSNR, SSIM, LPIPS) are measured outputs, not fitted parameters, and no quantity is defined in terms of another quantity from the same table. The reported GroupNorm result (A3 PSNR 29.38 vs. baseline 29.37) actually contradicts the abstract's claim that LayerNorm is important, which is an internal-consistency or experimental-stability issue (single runs, no reported seeds), not circularity. Likewise, citing [1] for the NAFNet design is a legitimate external source for the baseline, and the paper does not invoke any uniqueness theorem or self-citation chain to force conclusions. No circular step is present.
Assumptions & free parameters
free parameters (5)
- NAFNet block count and channel widths =
not reported
- Learning rate schedule and early stopping threshold =
not reported
- Blur sigma range =
[0, 3]
- Noise sigma range =
[0, 30]
- Random seed =
not reported
assumptions (4)
- domain assumption The NAFNet block from Chen et al. [1] is a valid baseline for image restoration
- domain assumption Synthetic Gaussian blur and noise are representative degradations for studying restoration design choices
- domain assumption PSNR, SSIM, and LPIPS adequately measure restoration quality
- ad hoc to paper A single training run per variant is sufficient to rank performance
Cite this review
Pith. "Pith review of A Comparative Study of NAFNet Baselines for Image Restoration." pith.science (2026). https://pith.science/paper/JOHW6REO
@misc{pith2026250619845,
author = {Pith},
title = {Pith review of: A Comparative Study of NAFNet Baselines for Image Restoration},
year = {2026},
howpublished = {\url{https://pith.science/paper/JOHW6REO}},
note = {Machine review of arXiv:2506.19845}
}
read the original abstract
We study NAFNet (Nonlinear Activation Free Network), a simple and efficient deep learning baseline for image restoration. By using CIFAR10 images corrupted with noise and blur, we conduct an ablation study of NAFNet's core components. Our baseline model implements SimpleGate activation, Simplified Channel Activation (SCA), and LayerNormalization. We compare this baseline to different variants that replace or remove components. Quantitative results (PSNR, SSIM) and examples illustrate how each modification affects restoration performance. Our findings support the NAFNet design: the SimpleGate and simplified attention mechanisms yield better results than conventional activations and attention, while LayerNorm proves to be important for stable training. We conclude with recommendations for model design, discuss potential improvements, and future work.
Figures
Reference graph
Works this paper leans on
-
[2]
Image denoising by sparse 3-d transform-domain collaborative filtering
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing , 16(8):2080--2095, 2007
work page 2007
-
[3]
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 7132--7141, 2018
work page 2018
-
[4]
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, Canada, 2009
work page 2009
-
[5]
Swinir: Image restoration using swin transformer
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops , pages 1833--1844, 2021
work page 2021
-
[6]
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing , 13(4):600--612, 2004
work page 2004
-
[7]
Yuxin Wu and Kaiming He. Group normalization. In European Conference on Computer Vision (ECCV) , pages 3--19, 2018
work page 2018
-
[8]
Restormer: Efficient transformer for high-resolution image restoration
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5728--5739, 2022
work page 2022
-
[9]
Multi-stage progressive image restoration
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14821--14831, 2021
work page 2021
Show all 10 references
-
[10]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 586--595, 2018
2018
-
[11]
11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...
2022 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.