REVIEW 5 major objections 4 minor 20 references
Machine-learning enables Image Reconstruction and Classification in a "see-through" camera
T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A convolutional network trained on sensor patterns from a see-through, lensless camera can reconstruct the original displayed characters with good fidelity for simple datasets, and classification from those reconstructed images can rival…
desk verdict Unfinished draft whose central reconstruction claim is undermined by the reported generalization gap; the raw-vs-reconstructed comparison is worth a look only after major completion. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the U-net encoder-decoder: convolutional stages halve the spatial dimensions while doubling the channel count, decoder stages reverse that process, and skip connections pass encoder features to the decoder so fine detail is not lost. The network is augmented with dense blocks and trained with pixel-wise cross-entropy loss on sigmoid outputs, a choice that encourages sparse reconstructions. Because the camera's point-spread function is space variant—the blur at the sensor depends on where the object pixel came from—no single convolution kernel can undo the blur, so the network must learn a many-to-one mapping from sensor patterns back to displayed images. The classification comparison uses the same off-the-shelf classifier on raw sensor data and on reconstructed images, testing whether the reconstructed image is a useful intermediate representation rather than just a visually cleaner picture.
What would settle it
Present the trained reconstructor with a displayed image it never trained on—for example, MNIST digits 6 through 9 if training used only digits 0 through 5—at the same geometry and sensor. If those unseen digits come out unrecognizable and the mean absolute error jumps far above the reported MNIST test value of 0.1014, the network has memorized the training classes rather than learned the camera's inverse; if unseen digits still reconstruct, the general-inverse claim survives.
Extended reading notes
Core claim
On the paper's own terms, the finding is that a U-net-based convolutional network can learn an inverse mapping for a lensless camera whose blur changes across the field of view: for the six MNIST classes the training mean absolute error is 0.0129 and the testing error 0.1014; for the 47-class EMNIST set the values are 0.0730 and 0.1213; for the 49-class Kanji49 set they are 0.0994 and 0.1786. The classification comparison shows that for MNIST, classifying reconstructed images gives a testing accuracy of 0.809, above the 0.612 obtained from raw sensor data; for EMNIST, reconstruction lifts testing accuracy from 0.019 to 0.397; and for Kanji49, it drops from 0.433 to 0.308. The authors conclude that similar classification accuracy is likely achievable with either route given further network optimization, and that reconstruction is demonstrated with good fidelity for MNIST in particular.
Load-bearing premise
The central assumption is that a network trained and tested only on the three label sets, all displayed on the same LCD at the same 250 mm distance and captured by the same sensor, has learned a general inverse of the camera's blur rather than a per-class lookup.
Editorial extensions
If this is right
- On a fixed display-sensor geometry, a see-through camera can act as an image-forming device for a restricted set of characters, turning a transparent window into a lensless display reader.
- Classifying reconstructed images instead of raw sensor patterns can improve accuracy on simple label sets—MNIST testing accuracy rises from 0.612 to 0.809—so reconstruction can be a useful preprocessing step when the label space is small.
- The benefit of reconstruction disappears on the largest, most variable dataset tested (Kanji49: reconstructed 0.308 vs raw 0.433), so reconstruct-then-classify is not generally superior to direct classification from sensor data.
- Any practical deployment inherits the paper's fixed conditions: same LCD, same 250-mm geometry, same sensor, and same alphabets; changing any of these would require retraining or further experiments.
Reading between the lines
- A direct test of whether the network learned a true inverse rather than a label-set lookup would be to keep the camera fixed and reconstruct an image unlike anything in training, such as a handwritten digit from a class excluded from training or a photograph of a non-character.
- The pattern across the three datasets suggests that as the label set grows, end-to-end classification on raw sensor data may be the more scalable route, with reconstruction useful mainly when humans need to see the image.
- Because the camera blur is space variant, the trained network is likely tied to the exact window-sensor geometry; a testable extension is to train at one object distance and evaluate at another to see how far the learned inverse generalizes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes a computational "see-through" camera, consisting of a transparent plexiglass window and a bare CMOS sensor, and proposes to reconstruct displayed images from the raw sensor measurements with a U-Net-style convolutional network. The authors report reconstruction MAE and classification accuracy for three datasets (MNIST restricted to 6 classes, EMNIST with 47 classes, and Kanji49 with 49 classes), comparing classification on original images, raw sensor images, and reconstructed images. The paper's central claim, stated in the conclusion, is that a U-net-based network can reconstruct MNIST images with good fidelity, and the abstract generalizes this to "image reconstruction can be achieved via a convolutional neural network for a see-through computational camera." The manuscript is presented as a preliminary proof of concept rather than a full system demonstration.
Significance. If the central claim were established, the paper would add a data point to the growing body of work on learned inverses for lensless and computational cameras, and it would provide a useful comparison between classification directly on raw sensor data and classification after learned reconstruction. The authors are honest about the preliminary nature of the work and use publicly available datasets. However, the significance is substantially undercut by the incomplete state of the manuscript and by the lack of evidence that the learned reconstruction generalizes beyond the closed label sets used in training. The paper explicitly notes the absence of a completed conclusions section and contains duplicated and garbled passages, which makes it unsuitable for publication in its present form.
major comments (5)
- [Section 5 (Conclusions)] The manuscript is an incomplete draft: Section 5 is literally the placeholder text "...TO D O" and the reference list is cut off mid-list. This is not a presentation issue that can be fixed with light editing; the paper lacks a proper conclusion and a complete reference list, and several paragraphs (e.g., the duplicated dataset description in Section 3 and the duplicated figure captions) are garbled. A journal submission must be a complete document.
- [Section 3, Table 1 and Figure 6] The central generalization claim is underdetermined by the reported experiments. All training and test images come from closed label sets (e.g., 6 MNIST classes) displayed on the same LCD at the same ~250 mm geometry. The observed training-to-test MAE gap is large (MNIST: 0.0129 training vs. 0.1014 testing, nearly 8x), and the reconstructed-image classifier drops from 99.7% training accuracy to 80.9% testing accuracy. These gaps are consistent with the network learning label-set-specific templates rather than a general inverse of the camera's space-variant PSF. The paper provides no out-of-distribution test (e.g., unseen classes, different geometry, or different sensor) that would distinguish these hypotheses, so the central claim that a U-net learns a general inverse mapping is not supported.
- [Section 4.2 / Figure 6 and Table 2] The Kanji49 results directly contradict the abstract's claim that "similar classification accuracy is likely possible in both cases." For Kanji49, raw-sensor testing accuracy (43.33%) is higher than reconstructed-image testing accuracy (30.77%), a substantial and unexplained reversal. The paper's explanation ("further parameter tuning") is unfalsifiable as stated and does not address the fact that reconstruction is demonstrably hurting classification on this dataset. This finding weakens the paper's comparative claim about raw versus reconstructed classification.
- [Section 2 (Network and Training) and Section 3] The empirical results are reported without error bars, repeated trials, or any measure of variability. Table 1 and Figure 6 report single runs with no indication of the number of seeds, no standard deviation, and no statistical comparison between methods. Given the small generalization gap for some datasets and the large gap for others, the authors should report variance across independent training runs or justify why a single run is sufficient.
- [Section 4.2 / raw sensor resolution] The comparison between raw-sensor classification and reconstructed-image classification is confounded by different input preprocessing: raw sensor images are resized to 125x170 pixels, while reconstructed images are resized to 32x32 pixels. This is a 22x difference in pixel count and may independently affect classifier performance, making the raw-versus-reconstructed comparison difficult to interpret. The paper does not discuss this confound.
minor comments (4)
- [Section 1, Fig. 1 caption] The caption says "The letter 'o' from the MNIST dataset" but MNIST contains digits, not letters; this is likely a typo for a handwritten digit.
- [Section 2, loss function] The pixel-wise cross-entropy loss is introduced as a sparsity-inducing loss for binary pixels, but the ground-truth images are grayscale, not binary. The manuscript should clarify whether the ground truth is binarized before training or how the cross-entropy is applied to continuous intensities.
- [Section 3, duplicated text] The paragraph beginning "For each data set, we trained and tested 3 classification networks..." appears twice with slightly different wording, and the garbled section "EM NI ST is the an augm entation of M NI ST..." is a corrupted duplicate of the dataset description. These should be cleaned up.
- [References] The reference list is incomplete; it ends at reference [9] despite references numbered up to [20] in the text, and some citations (e.g., ref. [15]) appear to be incorrectly numbered relative to the in-text callouts.
Circularity Check
No significant circularity: the paper reports an empirical train/test machine-learning pipeline, not a derived prediction, and its self-citations are contextual rather than load-bearing.
full rationale
This paper makes no first-principles derivation claim; it reports a convolutional neural network trained on measured sensor images from a see-through camera and evaluated on a disjoint 9:1 test split. The reconstruction network is trained with pixel-wise cross-entropy on ground-truth LCD images, and test MAE values are reported separately from training MAE, so the reported testing performance is not fitted input renamed as prediction. The classifier used to evaluate reconstructed images is a second trained network, which is a standard evaluation procedure and not a circular step. The authors cite their own prior work for the hardware platform ([8], [10]) and for prior lensless-camera machine-learning results ([9]), but the present experiments are independently measured and the cited works are used as background, not as the justification for the current result. No uniqueness theorem, ansatz, or derived identity is imported from self-citations. Concerns about generalization to unseen classes or geometries are correctness/robustness issues, not circularity. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (5)
- Learning rate =
0.001
- Training epochs =
50 (reconstruction), 30 (classification)
- Gaussian noise variance for data augmentation =
0.001
- Crop and resize targets =
128x128 for reconstruction, 32x32 for reconstructed classification, 125x170 for raw sensor classification
- Train/test split ratio =
9:1
assumptions (5)
- domain assumption The sensor data is a deterministic, space-variant transformation of the displayed image, stable across the experiment.
- domain assumption The 9:1 random split is sufficient to test generalization for each dataset.
- domain assumption Pixel-wise cross-entropy with sigmoid is a suitable reconstruction loss for grayscale images.
- domain assumption SimpleNet classification accuracy is a valid metric for reconstruction fidelity.
- domain assumption The chosen label sets (MNIST 6, EMNIST 47, Kanji49 49) are representative enough to support conclusions about general lensless imaging.
Cite this review
Pith. "Pith review of Machine-learning enables Image Reconstruction and Classification in a "see-through" camera." pith.science (2026). https://pith.science/paper/AQR3JXI4
@misc{pith2026190809401,
author = {Pith},
title = {Pith review of: Machine-learning enables Image Reconstruction and Classification in a "see-through" camera},
year = {2026},
howpublished = {\url{https://pith.science/paper/AQR3JXI4}},
note = {Machine review of arXiv:1908.09401}
}
read the original abstract
We demonstrate that image reconstruction can be achieved via a convolutional neural network for a "see-through" computational camera comprised of a transparent window and a CMOS image sensor. Furthermore, we compared classification results using a classifier network for the raw sensor data vs the reconstructed images. The results suggest that similar classification accuracy is likely possible in both cases with appropriate network optimizations. All networks were trained and tested for the MNIST (6 classes), EMNIST and the Kanji49 datasets.
Figures
Reference graph
Works this paper leans on
-
[1]
P. Wang and R. Menon, “Ultra-high sensitivity color imaging via a transparent diffractive-filter array and computational optics,” Optica 2(11) 933-939 (2015)
work page 2015
-
[2]
Computational multi-spectral video imaging,
P. Wang and R. Menon, “Computational multi-spectral video imaging,” J. Opt. Soc. Am. A 35(1), 189-199 (2018)
work page 2018
-
[3]
Computational snapshot angular-spectral lensless imaging
P. Wang and R. Menon, “Computational snapshot angular-spectral lensless imaging,” arXiv preprint arXiv:1707.08104 [physics.optics] (2017)
work page Pith review arXiv 2017
-
[4]
Deep-brain imaging via epi-fluorescence computational cannula microscopy,
G. Kim, N. Nagarajan, E. Pastuzyn, K. Jenks, M. Capecchi, J. Sheperd and R. Menon, “Deep-brain imaging via epi-fluorescence computational cannula microscopy,” Sci. Rep., 7, 44791 (2017)
work page 2017
-
[5]
Numerical analysis of computational cannula microscopy,
G. Kim and R. Menon, “Numerical analysis of computational cannula microscopy,” Appl. Opt. 56(9), D1-D7 (2017)
work page 2017
-
[6]
Cannula-based computational fluorescence microscopy,
G. Kim, N. Nagarajan, M. Capecchi and R. Menon, “Cannula-based computational fluorescence microscopy,” Appl. Phys. Lett. 106, 261111 (2015)
work page 2015
-
[7]
An ultra-small 3D computational microscope,
Ganghun Kim and R. Menon, “An ultra-small 3D computational microscope,” Appl. Phys. Lett. 105 061114 (2014)
work page 2014
-
[8]
Lensless photography with only an image sensor,
G. Kim, K. Isaacson, R. Palmer and R. Menon, “Lensless photography with only an image sensor,” Appl. Opt. 56(23),6450-6456 (2017)
work page 2017
Show all 20 references
-
[9]
Lensless-camera based machine learning for image classification,
G. Kim, S. Kapetanovic, R. Palmer and R. Menon, “Lensless-camera based machine learning for image classification,” arXiv preprint arXiv:1709.00408 [cs.CV] (2017)
2017 arXiv
-
[10]
Computational imaging enables a “see-through
G. Kim and R. Menon, “Computational imaging enables a “see-through” lensless camera,” Opt. Exp. 26(18) 22826-22836 (2018)
2018
-
[11]
On the use of deep learning in computational imaging,
G. Barbastathis, A. Ozcan and G. Situ, “On the use of deep learning in computational imaging,” Optica 6(8) 921-943 (2019)
2019
-
[12]
U-net: Convolutional networks for biomedical image segmentation,
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015
2015
-
[13]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778 (2016)
2016
-
[14]
What uncertainties do we need in bayesian deep learning for computer vision?,
A. Kendall and Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?,” In Advances in neural information processing systems, p. 5574–5584 (2017)
2017
-
[15]
Risk-sensitive loss functions for sparse multi-category classification problems,
S. Suresh, N. Sundararajan, and P. Saratchandran, “Risk-sensitive loss functions for sparse multi-category classification problems,” Information Sciences 178(12):2621–2638 (2008)
2008
-
[16]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, (2014)
2014 arXiv
-
[17]
Lets keep it simple, using simple architectures to outperform deeper and more complex architectures,
S. H. Hasanpour, M. Rouhani, M. Fayyaz, and M.Sabokrou, “Lets keep it simple, using simple architectures to outperform deeper and more complex architectures,” arXiv preprint arXiv:1608.06037 (2016)
2016 arXiv
-
[18]
LeCun and C
Y. LeCun and C. Cortes. MNIST handwritten digit database (2010)
2010
-
[19]
Emnist: an extension of mnist to handwritten letters,
G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “Emnist: an extension of mnist to handwritten letters,” arXiv preprint arXiv:1702.05373 (2017)
2017 arXiv
-
[20]
Deep learning for classical japanese literature,
T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha, “Deep learning for classical japanese literature,” arXiv preprint arXiv: 1812.01718v1 (2018)
2018 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.