REVIEW 5 major objections 4 minor 38 references
The RealDefocus Benchmark for Defocus Deblurring
T0 review · 5 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Training on the RealDefocus benchmark — a large real-world set of aperture pairs — improves single-image defocus deblurring across every architecture tested, with cross-dataset PSNR gains up to about 1.4 dB on the external RealDOF set.
desk verdict RealDefocus is a useful benchmark extension, but the headline cross-dataset gains are undercut by borrowed baselines and undisclosed pretraining. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the paired real-aperture capture: each scene is photographed at multiple f-stops, with f/22.0 designated as the sharp ground truth and smaller apertures (f/2.0, f/4.0, f/8.0) as defocused inputs. This reverses the original bokeh-rendering data into well-aligned real defocus pairs across a wide blur range. The benchmark adds fixed train/validation/test splits and a cross-dataset validation protocol using RealDOF as an external test set; that protocol is what turns the data collection into a reproducible benchmark.
What would settle it
Construct or obtain a set of scenes with true all-in-focus references (for example, via focus stacking) and evaluate RealDefocus-trained models against those references instead of against f/22 images. If the cross-dataset advantage over DPDD-S-trained models shrinks or disappears, the reported gains are partly an artifact of the f/22 target. A simpler check is to measure residual blur in f/22 images by comparing them to focus-stacked versions of the same scenes.
Extended reading notes
Core claim
The central claim is that RealDefocus provides a more effective real-world training signal for single-image defocus deblurring than DPDD-S, and that this advantage transfers to unseen data. On the RealDOF evaluation set, all ten evaluated architectures improve in PSNR and SSIM and reduce LPIPS when trained on RealDefocus instead of DPDD-S. The authors attribute the gains to RealDefocus's 23,000 image pairs across 4,400 scenes, apertures spanning f/2.0 to f/20, and 6000x4000 resolution, which yields stronger defocus blur and wider scene diversity than the fixed-aperture, smaller DPDD-S set.
Load-bearing premise
The entire benchmark rests on the assumption that the f/22.0 capture is a sufficiently sharp, all-in-focus ground truth; the paper itself concedes in Section 5 that aperture-based capture leaves residual defocus in the ground truth.
Editorial extensions
If this is right
- RealDefocus becomes a natural default training resource for SIDD, replacing the smaller, fixed-aperture DPDD-S in supervised setups.
- Standardized splits and a unified protocol make published deblurring numbers more directly comparable across architectures.
- The f/2.0 regime, with the strongest defocus blur, becomes a challenging stress test that separates methods on hard, real-world cases.
- Bokeh-rendering models such as Bokehlicious transfer well to deblurring, suggesting the two tasks share useful inductive structure.
- The consistent cross-dataset gains imply that dataset scale and aperture coverage, not just architecture, drive real-world deblurring performance.
Reading between the lines
- Because the f/22 reference still contains residual defocus, some of the reported PSNR gains may reflect better matching to an f/22 image rather than to a true all-in-focus latent; a focus-stacked reference would isolate that effect.
- The same aperture-pair data could support related tasks such as depth-from-defocus or controllable bokeh rendering, since the benchmark already spans a continuous aperture range.
- The uniformity of gains across architectures suggests a testable rule: when selecting training data for real defocus blur, prefer many real aperture pairs over a smaller specialized sensor dataset.
- A natural extension is to use RealDefocus as a pretraining set and then fine-tune on RealDOF; the paper does not report this, but the cross-dataset improvements make it a likely next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents RealDefocus, a benchmark for single-image defocus deblurring derived from the RealBokeh dataset by reversing the roles of blurred input and sharp ground truth. It defines training/validation/test splits, evaluates ten architectures across four method categories on RealDefocus, and reports a cross-dataset validation study on RealDOF. The central claim is that training on RealDefocus improves cross-dataset generalization compared to training on DPDD-S, with consistent PSNR/SSIM/LPIPS gains across all architectures.
Significance. If the claims are established, RealDefocus would be a valuable large-scale real-world resource for SIDD, addressing the scarcity of high-resolution defocus pairs and providing a standardized evaluation protocol. The paper's strengths include a public dataset and splits, evaluation across a diverse set of recent architectures, and an external cross-dataset validation on RealDOF. However, the evidence as presented is not fully controlled: the DPDD-S comparison mixes in borrowed baseline numbers, the comparison is confounded by dataset scale, and the ground-truth quality is explicitly conceded to be imperfect. These issues need to be resolved before the central claim can be accepted as stated.
major comments (5)
- [Section 4.2, Table 3] The caption states that 'Some DPDDS-trained RealDOF results are taken from [14, 22, 29].' Section 3.2 specifies a training protocol for RealDefocus (2000x1500 resolution, 512x512 patches, flips/rotations, per-method losses), but it does not state that the borrowed DPDD-S baselines were trained under the same protocol. Differences in training resolution, patch size, iteration count, optimizer schedule, loss weights, or evaluation cropping can shift PSNR by tenths of a dB, which is the same magnitude as the reported gains (+0.25 to +1.4 dB). The comparison is therefore not controlled. Please retrain all methods on DPDD-S under the identical protocol, or report the exact protocol for the borrowed numbers and justify comparability.
- [Section 4.2, Table 3 vs. Table 1] The comparison between RealDefocus and DPDD-S is confounded by dataset scale: RealDefocus has 20,500 training samples while DPDD-S has only 350. The consistent gains in Table 3 may simply reflect the 58x larger training set, not the claimed 'increased scale, scene diversity, and aperture coverage' as drivers of generalization. To support the causal attribution in the conclusion, the authors should include a matched-size ablation (e.g., a random 350-image subset of RealDefocus, or DPDD-S with repeated/extra data) or rephrase the claim to state that the current experiment does not separate scale from content.
- [Sections 2.2 and 5] The paper calls the f/22 image the 'sharp ground truth' (Fig. 1, Sec. 2.2), but Section 5 explicitly concedes that 'aperture-based capture protocols... leave residual defocus in the ground truth.' This residual blur biases all absolute metrics in Table 2 and undermines the benchmark's premise of providing all-in-focus references. Please quantify the residual defocus (e.g., edge-width statistics or comparison with a focus-stacked reference on a subset) or explicitly redefine the task as 'recovery of the f/22-acquired image' rather than all-in-focus restoration.
- [Table 2, Input row] For the raw input, LPIPS at f/4.0 is reported as 0.4028, which is higher than the LPIPS at f/2.0 (0.3238), even though f/2.0 produces stronger defocus blur. This inversion is physically unexpected and suggests a typo or inconsistent evaluation across aperture columns. Please verify the numbers and correct or explain the discrepancy.
- [Sections 3.2 and 4.2, Bokehlicious row] Bokehlicious originates from the same RealBokeh collection as RealDefocus and was originally trained for bokeh rendering. The paper does not state whether the 'trained on RealDefocus' Bokehlicious entry is initialized from the original checkpoint. If so, the model has already seen the same scenes' shallow-depth-of-field images, which constitutes data leakage and makes the +1.402 dB gain in Table 3 uninterpretable. Please explicitly describe the initialization and training procedure for this entry, and also disclose the authors' relationship to dataset [1] in the main text.
minor comments (4)
- [Figure 2] The per-image PSNR values in Figure 2 differ markedly from the average values in Table 2 (e.g., DRBNet f/2.0: 25.28 in Fig. 2 vs. 22.162 in Table 2). Please clarify that Figure 2 reports per-example PSNR, not the benchmark average, to avoid confusion.
- [Section 3.2] The training protocol lists resolution and patch size but omits iteration counts, number of training epochs, batch size, hardware, and number of runs. Adding these details would improve reproducibility.
- [Table 2, last column] The 'all f-stops' column should specify how the average is weighted across aperture settings (e.g., equal weight per f-stop or per image), since the dataset distribution may be imbalanced.
- [Section 4.2] RealDOF contains only 50 test images. Given the small evaluation set, it would be helpful to report variance or a significance test across multiple training runs, or at least acknowledge the limited sample size.
Circularity Check
No significant circularity: the cross-dataset claim rests on external RealDOF/DPDD-S benchmarks; self-citations to [1] are transparent dataset/model origins, not derived predictions.
full rationale
The paper's central claim—that training on RealDefocus improves cross-dataset generalization—is supported by Table 3, which evaluates models on the external RealDOF dataset [19] after training on either external DPDD-S [10] or RealDefocus [1]. RealDOF and DPDD-S are not constructed from the paper's own outputs, so the comparison is not circular by construction. RealDefocus itself is inherited from the authors' prior RealBokeh work [1], but this is an explicit data-source statement, not a prediction or fitted parameter; the f/22 reference is acknowledged in Section 5 to contain residual defocus, which is a ground-truth validity limitation rather than a circular reduction. The inclusion of Bokehlicious [1], a model from the authors' prior work, is likewise transparent and does not make the cross-dataset deltas equal to a fitted input. The caption's disclosure that some DPDD-S-trained RealDOF results are taken from [14,22,29] raises a legitimate experimental-protocol concern, but it does not show that the reported gains are identical to the inputs by definition. No equation, definition, or citation chain reduces the central claim to its own assumptions, so no circularity step is established.
Assumptions & free parameters
assumptions (5)
- domain assumption Images captured at f/22.0 are a sufficiently sharp, all-in-focus ground truth.
- domain assumption Reversing RealBokeh pairs yields valid SIDD training pairs (defocused input -> sharp output) with well-aligned apertures.
- domain assumption The predefined training/validation/test splits are leak-free and representative.
- domain assumption PSNR/SSIM/LPIPS, measured against f/22 references, adequately quantify restoration quality and generalization.
- domain assumption DPDD-S-trained RealDOF baselines borrowed from [14,22,29] are directly comparable to new RealDefocus-trained runs in Table 3.
Cite this review
Pith. "Pith review of The RealDefocus Benchmark for Defocus Deblurring." pith.science (2026). https://pith.science/paper/4AUXU6CZ
@misc{pith2026260721078,
author = {Pith},
title = {Pith review of: The RealDefocus Benchmark for Defocus Deblurring},
year = {2026},
howpublished = {\url{https://pith.science/paper/4AUXU6CZ}},
note = {Machine review of arXiv:2607.21078}
}
read the original abstract
Single-Image Defocus Deblurring (SIDD) aims to recover an all-in-focus image from a single defocused observation, but rigorous and reproducible evaluation remains challenging due to the scarcity of realistic, high-resolution datasets with well-aligned defocused/sharp pairs and standardized protocols. We build on RealDefocus, a benchmark derived from the real-world RealBokeh dataset originally proposed for Bokeh Rendering. RealDefocus provides paired defocused inputs and sharp ground truth images, predefined training/validation/test splits, and a unified evaluation framework for comparing image restoration and neural rendering approaches. We further outline a benchmarking protocol with cross-dataset validation to assess reconstruction quality and generalization. The project page is publicly available at: www.github.com/TimSeizinger/RealDefocus-Benchmark.
Reference graph
Works this paper leans on
-
[1]
This phenomenon, commonly observed when us- ing large aperture settings or imaging scenes with substan- tial depth variation, is often associated with thebokeh ef- fect[1]
INTRODUCTION When capturing a scene with a finite depth of field, ob- jects outside the focal plane appear blurred due to optical defocus. This phenomenon, commonly observed when us- ing large aperture settings or imaging scenes with substan- tial depth variation, is often associated with thebokeh ef- fect[1]. While shallow depth of field can be intention...
-
[2]
RELA TED WORK 2.1. Defocus Deblurring Datasets One of the first datasets enabling quantitative evaluation was RTF [9], captured using a Lytro light field camera [16]. While innovative, RTF contains only 22 images at a low spa- tial resolution (360×360), with its defocus blur synthetically rendered from light field data. Consequently, its scale and realism...
arXiv 2026
-
[3]
BENCHMARK METHODOLOGY 3.1. Model Selection To provide a comprehensive and representative bench- mark, we evaluate methods from four complementary cate- gories: (i) dedicated Defocus Deblurring approaches, (ii) mo- tion deblurring methods, (iii) general-purpose image restora- tion architectures, and (iv) a Bokeh Rendering model. This selection reflects bot...
arXiv 1910
-
[4]
BENCHMARKING AND RESULTS ANALYSIS Overall, the benchmark aims to ensure fairness while re- specting the design principles of each method. By combining physics-driven defocus models, motion-deblurring state-of- the-art architectures, general-purpose restoration backbones, and a Bokeh Rendering model, we provide a broad and bal- anced evaluation of current ...
-
[5]
Due to a do- main gap, training with synthetic Defocus Blur leads to un- satisfying results under real world conditions [31]
DISCUSSIONS Current real-world datasets, including RealDefocus [1], remain limited by aperture-based capture protocols, which leave residual defocus in the ground truth, while light-field cameras are limited in spatial resolution [16]. Due to a do- main gap, training with synthetic Defocus Blur leads to un- satisfying results under real world conditions [...
-
[6]
CONCLUSION In this work, we introduce a benchmark protocol forRe- alDefocus, a large-scale real-world dataset for SIDD with standardized splits and evaluation protocols. Our results demonstrate that training on RealDefocus improves cross- dataset generalization versus prior datasets, while enabling unified comparison across task-specific and general resto...
-
[7]
Bokehlicious: Photoreal- istic bokeh rendering with controllable apertures,
Tim Seizinger, Florin-Alexandru Vasluianu, Marcos Conde, Zongwei Wu, and Radu Timofte, “Bokehlicious: Photoreal- istic bokeh rendering with controllable apertures,” inICCV, 2025
2025
-
[8]
What is ’bokeh’?,
John Kennerdell, “What is ’bokeh’?,”Photo Techniques mag- azine, vol. May/June, pp. 28, 1997
1997
Show all 38 references
-
[9]
End-to-end text recognition with convolutional neural net- works,
Tao Wang, David J Wu, Adam Coates, and Andrew Y Ng, “End-to-end text recognition with convolutional neural net- works,” inICPR. IEEE, 2012
2012
-
[10]
Text recognition in the wild: A survey,
Xiaoxue Chen, Lianwen Jin, Yuanzhi Zhu, Canjie Luo, and Tianwei Wang, “Text recognition in the wild: A survey,”ACM Computing Surveys (CSUR), vol. 54, no. 2, pp. 1–35, 2021
2021
-
[11]
You only look once: Unified, real-time object de- tection,
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi, “You only look once: Unified, real-time object de- tection,” inCVPR, 2016
2016
-
[12]
Analysis of the impact of lens blur on safety-critical automotive object detection,
Dara Molloy, Patrick M ¨uller, Brian Deegan, Darragh Mullins, Jonathan Horgan, Enda Ward, Edward Jones, Alexander Braun, and Martin Glavin, “Analysis of the impact of lens blur on safety-critical automotive object detection,”IEEE Access, vol. 12, pp. 3554–3569, 2024
2024
-
[13]
On clas- sification of distorted images with deep convolutional neural networks,
Yiren Zhou, Sibo Song, and Ngai-Man Cheung, “On clas- sification of distorted images with deep convolutional neural networks,” in2017 IEEE International conference on acous- tics, speech and signal processing (ICASSP). IEEE, 2017, pp. 1213–1217
2017
-
[14]
Image and depth from a conventional camera with a coded aperture,
Anat Levin, Rob Fergus, Fr ´edo Durand, and William T Free- man, “Image and depth from a conventional camera with a coded aperture,”ACM transactions on graphics (TOG), vol. 26, no. 3, pp. 70–es, 2007
2007
-
[15]
Non-parametric blur map regression for depth of field extension,
Laurent D’Andr `es, Jordi Salvador, Axel Kochale, and Sabine S¨usstrunk, “Non-parametric blur map regression for depth of field extension,”IEEE Transactions on Image Processing, vol. 25, no. 4, pp. 1660–1673, 2016
2016
-
[16]
Defocus deblur- ring using dual-pixel data,
Abdullah Abuolaim and Michael S Brown, “Defocus deblur- ring using dual-pixel data,” inECCV, 2020
2020
-
[17]
Learning to re- duce defocus blur by realistically modeling dual-pixel data,
Abdullah Abuolaim, Mauricio Delbracio, Damien Kelly, Michael S Brown, and Peyman Milanfar, “Learning to re- duce defocus blur by realistically modeling dual-pixel data,” inICCV, 2021
2021
-
[18]
Aifnet: All-in-focus image restoration network using a light field-based dataset,
Lingyan Ruan, Bin Chen, Jizhou Li, and Miu-Ling Lam, “Aifnet: All-in-focus image restoration network using a light field-based dataset,”IEEE Transactions on Computational Imaging, vol. 7, pp. 675–688, 2021
2021
-
[19]
Gaussian kernel mixture network for single image defocus deblurring,
Yuhui Quan, Zicong Wu, and Hui Ji, “Gaussian kernel mixture network for single image defocus deblurring,”Advances in Neural Information Processing Systems, vol. 34, pp. 20812– 20824, 2021
2021
-
[20]
Neumann network with recursive kernels for single image defocus deblurring,
Yuhui Quan, Zicong Wu, and Hui Ji, “Neumann network with recursive kernels for single image defocus deblurring,” in CVPR, 2023
2023
-
[21]
Quad-pixel image defocus deblurring: A new benchmark and model,
Hang Chen, Yin Xie, Xiaoxiu Peng, Lihu Sun, Wenkai Su, Xi- aodong Yang, and Chengming Liu, “Quad-pixel image defocus deblurring: A new benchmark and model,” inCVPR, 2025
2025
-
[22]
Ren Ng,Digital light field photography, stanford university, 2006
2006
-
[23]
Liff: Light field features in scale and depth,
Donald G Dansereau, Bernd Girod, and Gordon Wetzstein, “Liff: Light field features in scale and depth,” inCVPR, 2019
2019
-
[24]
Learn- ing to deblur using light field generated and real defocus im- ages,
Lingyan Ruan, Bin Chen, Jizhou Li, and Miuling Lam, “Learn- ing to deblur using light field generated and real defocus im- ages,” inCVPR, 2022
2022
-
[25]
Iterative filter adaptive network for sin- gle image defocus deblurring,
Junyong Lee, Hyeongseok Son, Jaesung Rim, Sunghyun Cho, and Seungyong Lee, “Iterative filter adaptive network for sin- gle image defocus deblurring,” inCVPR, 2021
2021
-
[26]
Restormer: Efficient transformer for high-resolution image restoration,
Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Restormer: Efficient transformer for high-resolution image restoration,” inCVPR, 2022
2022
-
[27]
Efficient visual state space model for image deblurring,
Lingshun Kong, Jiangxin Dong, Jinhui Tang, Ming-Hsuan Yang, and Jinshan Pan, “Efficient visual state space model for image deblurring,” inCVPR, 2025
2025
-
[28]
Revisiting image deblurring with an efficient convnet,
Lingyan Ruan, Mojtaba Bemana, Hans-peter Seidel, Karol Myszkowski, and Bin Chen, “Revisiting image deblurring with an efficient convnet,”arXiv preprint arXiv:2302.02234, 2023
2023 arXiv
-
[29]
Eamamba: Efficient all-around vision state space model for image restoration,
Yu-Cheng Lin, Yu-Syuan Xu, Hao-Wei Chen, Hsien-Kai Kuo, and Chun-Yi Lee, “Eamamba: Efficient all-around vision state space model for image restoration,”ICCV, 2025
2025
-
[30]
Efficient frequency domain-based transformers for high-quality image deblurring,
Lingshun Kong, Jiangxin Dong, Jianjun Ge, Mingqiang Li, and Jinshan Pan, “Efficient frequency domain-based transformers for high-quality image deblurring,” inCVPR, 2023
2023
-
[31]
Edge-based defocus blur estimation with adaptive scale selection,
Ali Karaali and Claudio Rosito Jung, “Edge-based defocus blur estimation with adaptive scale selection,”IEEE Trans- actions on Image Processing, vol. 27, no. 3, pp. 1126–1137, 2017
2017
-
[32]
Attention is all you need,
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017
2017
-
[33]
Efficiently mod- eling long sequences with structured state spaces,
Albert Gu, Karan Goel, and Christopher R ´e, “Efficiently mod- eling long sequences with structured state spaces,” inThe In- ternational Conference on Learning Representations (ICLR), 2022
2022
-
[34]
Mamba: Linear-time sequence modeling with selective state spaces,
Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,”arXiv preprint arXiv:2312.00752, 2023
2023 arXiv
-
[35]
Single image defocus de- blurring via implicit neural inverse kernels,
Yuhui Quan, Xin Yao, and Hui Ji, “Single image defocus de- blurring via implicit neural inverse kernels,” inICCV, 2023
2023
-
[36]
The unreasonable effectiveness of deep fea- tures as a perceptual metric,
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang, “The unreasonable effectiveness of deep fea- tures as a perceptual metric,” inCVPR, 2018
2018
-
[37]
Bokehme: When neural rendering meets classical rendering,
Juewen Peng, Zhiguo Cao, Xianrui Luo, Hao Lu, Ke Xian, and Jianming Zhang, “Bokehme: When neural rendering meets classical rendering,” inCVPR, 2022
2022
-
[38]
Extended depth-of-field via focus stacking and graph cuts,
Chao Zhang, John Bastian, Chunhua Shen, Anton van den Hengel, and Tingzhi Shen, “Extended depth-of-field via focus stacking and graph cuts,” in2013 IEEE International Confer- ence on Image Processing, 2013, pp. 1272–1276
2013
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.