DeepPD: Joint Phase and Object Estimation from Phase Diversity with Neural Calibration of a Deformable Mirror

Cedric Allier; Courtney Johnson; Diane Adjavon; Hari Shroff; Joren Husic; Larissa Heinrich; Magdalena C. Schneider; Patrick La Rivi\`ere; Stephan Saalfeld

REVIEW 2 major objections 6 minor 16 references

DeepPD: Joint Phase and Object Estimation from Phase Diversity with Neural Calibration of a Deformable Mirror

T0 review · 2 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DeepPD recovers both the unknown optical aberration and the object from only five phase-diversity images, by replacing the usual linear deformable-mirror calibration with a learned neural mirror model.

desk verdict DeepPD's learned deformable-mirror model is a genuine step forward for phase-diversity microscopy, but the self-referential training of that model is the load-bearing soft spot; the paper deserves peer review with a focus on independent wavefront validation. read the letter →

arxiv 2504.14157 v1 pith:BYB4HKOY submitted 2025-04-19 physics.optics cs.LG

Magdalena C. Schneider , Courtney Johnson , Cedric Allier , Larissa Heinrich , Diane Adjavon , Joren Husic , Patrick La Rivière , Stephan Saalfeld

show 1 more author

Hari Shroff

This is my paper · ORCID

classification physics.opticscs.LG

keywords phasediversitydeformablemirroradaptiveopticsfluorescencemicroscopyneuralrepresentationswavefrontsensingjointestimation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents DeepPD, a method that jointly estimates the unknown optical aberration (phase) and the fluorescent object from only five images: one aberrated image plus four deliberately aberrated diversity images. Its central move is to replace the standard linear calibration of a deformable mirror, which assumes the mirror's wavefront response is a sum of fixed influence functions, with a trained neural network that predicts the actual wavefront from the applied actuator voltages. Because the diversity phases enter the reconstruction model directly, a more faithful mirror model keeps the optimization from being misled, and the paper reports improved object and phase estimates compared with earlier approaches. A sympathetic reader would care because this moves phase-diversity imaging closer to practical biological use: fewer exposures, less phototoxicity, and joint aberration correction and deconvolution without a guidestar.

What carries the argument

The load-bearing mechanism is the learned deformable-mirror model, specifically a voltage-to-phase neural network that maps the 52 actuator voltages to a predicted phase map over the pupil. It is trained together with a reversed phase-to-voltage network using a cycle-consistency loss, so that voltage-to-phase-to-voltage and phase-to-voltage-to-phase round trips return the original inputs. This learned model replaces the linear influence-function calibration that prior phase-diversity methods rely on, and it is what lets the forward model capture nonlinear mirror effects and higher-order aberrations beyond the first 21 Zernike modes. The mirror model is bootstrapped: initial phase estimates come from the same neural-representation reconstruction, then the model is retrained for three cycles using its own predicted diversity phases.

What would settle it

Measure the mirror's actual wavefront with an external sensor, such as a pupil-matched interferometer or Shack-Hartmann sensor, across the same random voltage range used in training, and compare those measurements with the voltage-to-phase model's predictions. If the learned model's error is not smaller than the linear influence-function model's error by roughly the margin claimed for the reconstruction improvements, the central claim that the learned calibration drives the gains is not supported.

Watch

Extended reading notes

Core claim

DeepPD's central claim is that a single framework can jointly estimate the unknown phase aberration $\phi$ and the fluorescent object $O$ from exactly five images—one aberrated image plus four astigmatism phase-diversity images—by simulating the imaging equation $I_k = O * |\mathcal{F}^{-1}(P e^{i(\phi+\psi_k)})|^2$ and minimizing the mismatch between simulated and acquired images. The object and phase are represented by neural networks rather than pixel arrays, so the phase estimate is not confined to a low-order Zernike basis. The decisive addition is a learned mirror model: a neural network that predicts the actual wavefront produced by the deformable mirror from the 52 applied actuator voltages, replacing the assumption that the mirror response is a linear superposition of influence functions. Trained jointly with an inverse phase-to-voltage network and cycle-consistency losses, this model yields diversity phases that match the real mirror, and the paper reports that the resulting object and phase estimates are more accurate than those of Gauss-Newton, Poisson, and linear-mirror neural-representation baselines, with reliable wavefront recovery up to roughly 350 nm RMS wavefront error.

Load-bearing premise

The learned mirror model is trained using phase estimates produced by the same phase-diversity reconstruction approach it is meant to improve, so if those estimates carry systematic errors, the calibrated mirror model is only self-consistent with the algorithm rather than physically accurate.

Editorial extensions

If this is right

Only five acquired images (one aberrated plus four astigmatism-diversity images) are needed for joint object and phase recovery, a substantial reduction from roughly 100 images used by earlier neural-representation phase-diversity schemes.
Wavefront recovery is reported to be reliable up to about 350 nm RMS error, extending the previously reported range of about 200 to 250 nm for phase-diversity approaches.
Because the object is estimated jointly with the phase, the output is an aberration-corrected, deconvolved image even when no guidestar or unaberrated reference is available.
The trained phase-to-voltage network can later be used to apply desired phase diversities, and ultimately phase corrections, at acquisition time.
At high aberration magnitudes (roughly above 100 nm RMS wavefront error), the recovered-object quality metrics exceed those of the Gauss-Newton and Poisson baselines, while remaining comparable at low aberration magnitudes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the mirror model is trained in an iterative self-bootstrap from its own reconstructions, the same procedure could in principle be applied to any repeatable optical element whose wavefront response drifts, such as a spatial light modulator, without requiring an external wavefront sensor.
A closed-loop use of the trained phase-to-voltage network, applying the estimated aberration correction at acquisition time on moving samples, is a natural next step that the paper mentions but does not demonstrate.
The 121-dataset comparison suggests a threshold near 100 nm RMS below which the learned mirror model adds little, and above which its advantage grows; a targeted study varying aberration composition rather than just magnitude would clarify whether high-order modes drive the gain.
If the phase estimates used as training labels carry systematic biases from the reconstruction algorithm itself, the learned mirror model may be self-consistent with DeepPD rather than physically accurate; an independent wavefront comparison over the full voltage range would settle this directly.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

Quick take: DeepPD does something real—it replaces the standard linear influence-function model of the deformable mirror with a learned voltage-to-phase network inside a joint phase-and-object reconstruction from five phase-diversity images. The idea is straightforward, and the empirical comparison against the linear-mirror variant (NNRs) is the right control: if the learned mirror model didn't help, the comparison would show it. The gains at high aberration magnitudes are credible, and the paper is honest that quantitative Shack-Hartmann validation is limited and that the mirror calibration is only reliable within the range the phase-diversity analysis can retrieve.

The genuinely new piece is the mirror model training with cycle consistency. The rest—neural representations from NeuWS and the four astigmatism diversities from Johnson et al.—is a sensible combination, not a leap. The methods are described clearly enough to reproduce, and the 121-dataset comparison gives some weight to the robustness claim.

The soft spots are the ones you'd expect. Most load-bearing: the mirror model's training labels are phase estimates produced by the same phase-diversity analysis that DeepPD is meant to improve. The three-cycle retraining can converge to a self-consistent but physically wrong fixed point. The Shack-Hartmann comparison in Fig. 2d is qualitative, and the authors explicitly say quantitative comparison provides limited value. That's the weakest link. The metrics in Fig. 5 are reported as distributions without error bars or statistical tests, and the reference image for SSIM/PCC/PSNR is the recorded unaberrated image, which is not ground truth. These are addressable: independent wavefront measurements on a subset of mirror training voltages, confidence intervals, and code release would firm it up.

That said, the circularity concern doesn't sink the paper. The comparison against NNRs is still informative: both use the same reconstruction machinery, and the only difference is the mirror model. If the learned model merely encoded the reconstruction's biases, it's not obvious those biases would consistently improve the DCT norm and high-frequency content. The improvement is credible, just not fully nailed down.

Who it's for: anyone working on phase diversity, adaptive optics, or computational microscopy. It's a solid methods paper that deserves serious peer review. I'd send it to a referee with a request to focus on the mirror model validation, not desk reject.

Recommendation: accept for peer review, conditional on the authors addressing the circularity concern with independent wavefront data or at least a clear statement of the limitation.

Referee Report

2 major / 6 minor

Summary. The paper presents DeepPD, a phase-diversity reconstruction framework that jointly estimates the object and phase aberration using neural representations of both quantities, with a learned deformable-mirror model replacing the usual linear influence-function calibration. The method uses five images: one aberrated image and four images with astigmatism phase diversities. The mirror model consists of voltage-to-phase and phase-to-voltage networks trained on 3,930 phase-diversity stacks recorded on bead samples, using phase estimates obtained from the same neural-representation phase-diversity analysis, refined over three cycles. The authors evaluate DeepPD on an Argo-HM calibration slide and on immunolabeled myosin in fixed PtK2 cells, comparing against Gauss-Newton, Poisson, and neural-representation baselines with a linear mirror model (NNRs). They report improved object-reconstruction metrics (SSIM, PCC, PSNR, DCT norm), particularly for RMS wavefront errors above roughly 100 nm, and qualitatively compare the learned mirror model to Shack-Hartmann measurements.

Significance. If the learned mirror model is physically accurate, DeepPD is a valuable contribution: it reduces the number of diversity images required for guidestar-free joint phase/object estimation, removes the Zernike-basis limitation of earlier methods, and explicitly addresses nonlinear deformable-mirror response. The writing is clear, the method is described in sufficient detail to be reproduced, and the authors compare against several relevant baselines. The central empirical claim, however, depends on the mirror model's physical accuracy, which is not quantitatively validated; the training labels for the mirror model come from the same reconstruction algorithm that DeepPD is meant to improve. Hence the current evidence supports a conditional assessment rather than a definitive one.

major comments (2)

[Sec. 4.3.1, Sec. 4.3.4] The mirror model is trained on phase estimates produced by the same phase-diversity reconstruction that DeepPD is designed to improve, and the three-cycle iterative retraining only re-analyzes the same training datasets with the previous mirror model. This can converge to a self-consistent but physically incorrect solution. The only external validation, Fig. 2d, is qualitative; the manuscript states that a quantitative SH comparison 'provides limited value' and that pupil diameters cannot be matched directly. Because the claimed advantage of DeepPD over the NNRs variant (linear mirror model) is the learned mirror model, this circularity is load-bearing. Please provide quantitative validation of the mirror model on held-out voltages against SH measurements (or a synthetic ground-truth phase) and report the phase error; this is needed to establish that the improvement over the linear model reflects physical accuracy rather than algorithmic self-consistency.
[Sec. 5 (Metrics) and Fig. 5] The quantitative comparison in Fig. 5 uses the recorded unaberrated image as the reference for SSIM, PCC, and PSNR, after convolving the object estimates with the microscope's PSF. The unaberrated image is not ground truth; it contains noise and possibly residual aberration, and the assumed PSF may not match the actual imaging system. This makes the reported differences in metrics difficult to interpret and could bias the comparison. Please evaluate on synthetic data with known ground truth and report error bars or confidence intervals for the metrics across the 121 datasets.

minor comments (6)

[Sec. 2] The sentence beginning 'whereas NeuWS requires around 100 images' should start with 'Whereas'.
[Sec. 4.5, Eq. (12)] Equation (12) has an unbalanced parenthesis in the DCT-norm definition; please check the formula.
[Fig. 2d] The caption says the SH measurements use a zonal representation, but no scale or colorbar is provided for the phase maps, making the qualitative comparison hard to assess.
[Sec. 4.2.6] The paper does not state the number of random trials or seeds used for the neural-representation reconstructions; adding this would help assess the stability of the reported metrics.
[Sec. 3] In the Discussion, 'that that leverages' has a duplicated word.
[Declarations] The code availability statement says code will be released upon publication; for reproducibility, consider providing a version at submission.

Circularity Check

2 steps flagged · score 6.0 of 10

Mirror-model calibration is learned from the same phase-diversity reconstruction it is meant to improve, making the phase-estimation gain over the linear-mirror baseline partially self-consistency.

fitted input called prediction [Section 4.3.1 (Training data), Section 4.3.3 (Loss), Section 4.3.4 (Iterative improvement)]
"As a ground truth for the phase aberration does not exist, we use phase aberration estimates retrieved by our phase-diversity analysis using the neural representations as a proxy."

The voltage-to-phase mirror model is trained with a supervision loss on phase estimates produced by the same neural-representation phase-diversity analysis that DeepPD is designed to improve. Section 4.3.3 defines Lsup as the MSE between 'the phase estimate obtained from the phase-diversity analysis of the training data' and the mirror model's predicted phase. The mirror model therefore learns to reproduce the reconstruction algorithm's own estimates, not an independently measured wavefront. When DeepPD is compared with the linear-mirror variant (NNRs), the improvement can reflect self-consistency between the mirror model and algorithm biases rather than physical wavefront accuracy. The only external check, the Shack-Hartmann comparison in Fig.
self definitional [Section 4.3.4 (Iterative improvement of the mirror model)]
"Therefore, we decided on an iterative scheme, where we first train the mirror model with the initial phase estimates, then reanalyze the training dataset with phase diversities predicted from the trained mirror model, and subsequently retrain the mirror model with the newly obtained phase estimates."

This loop makes the mirror model and the phase-diversity reconstruction mutually defining: the mirror model is trained to match the phase-diversity algorithm's phase estimates, and the phase-diversity algorithm is then re-run with the mirror model's predicted diversities. Convergence after three cycles means the two have reached a self-consistent fixed point, not that the wavefront agrees with an external reference. The Discussion confirms the limitation: 'the mirror model used for calibration will be most accurate for phase aberrations in a range that can be reliably estimated based on the phase-diversity images.' In other words, the calibration is anchored to the same algorithm it is supposed to improve, so the phase part of the central claim is partially circular by construction.

full rationale

The core reconstruction pipeline in Sec. 4.2 is self-contained: it minimizes an MSE loss between the five measured images and images simulated from neural representations of object and phase, and the object estimates are tested against known Argo-HM patterns. Those object comparisons, and the image-quality metrics in Fig. 5, are not circular in themselves. However, the paper's novelty claim rests on the learned deformable-mirror model (Sec. 2 and Sec. 4.3), and that model is trained on phase estimates produced by the same neural-representation phase-diversity analysis that DeepPD is designed to improve (Sec. 4.3.1). The three-cycle retraining (Sec. 4.3.4) makes the model and reconstruction mutually consistent rather than independently validated; the only external wavefront check, the Shack-Hartmann comparison, is qualitative by the authors' own statement. Consequently, the reported improvement of DeepPD over the linear-mirror variant (NNRs) is at least partially explained by fitting the mirror model to the reconstruction algorithm's biases, even though the object-reconstruction component retains independent empirical content. This is a partial circularity in a central component, so the score is 6 rather than lower. No self-citation chain is load-bearing here; the issue is the provenance of the calibration labels.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the learned mirror model being physically accurate, which in turn depends on the self-consistency of the phase estimates used for training. The listed free parameters are the design choices that shape the mirror model's training and the phase penalty. The axioms are the imaging model and the validity assumptions on the mirror model. No new physical entities are introduced.

free parameters (4)

Mirror model loss weights (Lsup phase/voltage: 1, 0.01; Lcc phase/voltage: 10, 0.1; Lreg: 1) = 1, 0.01, 10, 0.1, 1
Chosen without derivation in Sec. 4.3.3; these weights balance phase prediction, voltage prediction, cycle consistency, and regularization when training the mirror model.
Phase penalty threshold = 200 pi rad
Selected in Sec. 4.2.5 and 4.3.3 to penalize extreme phase values; the value is described as well above physically relevant aberrations but is not derived.
Number of iterative mirror-model cycles = 3
Sec. 4.3.4 states that training/fine-tuning for three cycles produced no further improvement; this stopping rule is empirical, not guaranteed by a criterion.
Random aberration amplitude range A for mirror training set = 0.1 to 0.8 micrometers
Sec. 4.3.1: Zernike coefficients are drawn from uniform [-A,A] with A varied between 0.1 and 0.8 micrometers; this range defines the domain of validity of the mirror model.

assumptions (5)

domain assumption The imaging model is isoplanatic: a single phase aberration phi applies across the entire field of view, so image formation is a convolution (Eq. 1).
Eqs. 1 and 5 assume a spatially invariant PSF; the method does not handle spatially varying aberrations, which can occur in widefield microscopy of thick samples.
domain assumption The learned voltage-to-phase network accurately predicts the actual wavefront for the voltages and environmental conditions encountered at test time.
The mirror model is trained on 3,930 bead datasets with random voltages and then used on unseen myosin data; its accuracy is only qualitatively validated against a Shack-Hartmann sensor, so the claimed improvement depends on this generalization.
ad hoc to paper The phase estimates used as training targets for the mirror model are sufficiently accurate to serve as ground truth.
Sec. 4.3.1: the training targets come from the authors' own neural-representation phase-diversity analysis, which itself assumes a mirror model; this self-referentiality is the main circularity risk.
domain assumption The four astigmatism phase diversities, taken from prior work, are sufficient to disambiguate object and phase for the aberration range considered.
The method uses a fixed set of four diversities chosen by Johnson et al.; the paper does not re-optimize them for the learned mirror model or test robustness to this choice.
standard math The Fourier optics forward model (Eqs. 5-6) accurately describes the microscope imaging chain.
The method assumes PSF formation via pupil function and OTF multiplication; unmodeled effects such as background fluorescence, stray light, or camera noise are not explicitly accounted for.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DeepPD: Joint Phase and Object Estimation from Phase Diversity with Neural Calibration of a Deformable Mirror." pith.science (2026). https://pith.science/paper/BYB4HKOY

@misc{pith2026250414157,
  author       = {Pith},
  title        = {Pith review of: DeepPD: Joint Phase and Object Estimation from Phase Diversity with Neural Calibration of a Deformable Mirror},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BYB4HKOY}},
  note         = {Machine review of arXiv:2504.14157}
}

read the original abstract

Sample-induced aberrations and optical imperfections limit the resolution of fluorescence microscopy. Phase diversity is a powerful technique that leverages complementary phase information in sequentially acquired images with deliberately introduced aberrations--the phase diversities--to enable phase and object reconstruction and restore diffraction-limited resolution. These phase diversities are typically introduced into the optical path via a deformable mirror. Existing phase-diversity-based methods are limited to Zernike modes, require large numbers of diversity images, or depend on accurate mirror calibration--which are all suboptimal. We present DeepPD, a deep learning-based framework that combines neural representations of the object and wavefront with a learned model of the deformable mirror to jointly estimate both object and phase from only five images. DeepPD improves robustness and reconstruction quality over previous approaches, even under severe aberrations. We demonstrate its performance on calibration targets and biological samples, including immunolabeled myosin in fixed PtK2 cells.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

[1]

Hampson, K. M. et al. Adaptive optics for high-resolution imaging. Nature Reviews Methods Primers (2021)

work page 2021
[2]

Poyneer, L. A. Scene-based shack-hartmann wave-front sensing: analysis and simulation. Appl. Opt. (2003)

work page 2003
[3]

Kang, I., Zhang, Q., Yu, S. X. & Ji, N. Coordinate-based neural representations for computational adaptive optics in widefield microscopy. Nature Machine Intelligence (2024)

work page 2024
[4]

Guo, M. et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. Nature Communications (2025)

work page 2025
[5]

Wijethilake, N. et al. DEEP-squared: deep learning powered De-scattering with Excitation Patterning. Light: Science & Applications (2023)

work page 2023
[6]

Feng, B. Y. et al. NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media. Science Advances (2023)

work page 2023
[7]

Johnson, C. et al. Phase-diversity-based wavefront sensing for fluorescence microscopy. Optica (2024)

work page 2024
[8]

& Jin, Z

Liu, D., Liu, H. & Jin, Z. Direct phase sensing via an end-to-end model under high-order aberrations. Journal of the Optical Society of America A (2025)

work page 2025

Show all 16 references

[9]

& Bifano, T

Haber, A. & Bifano, T. General approach to precise deformable mirror control. Optics Express (2021)

2021
[10]

& Booth, M

Wang, B. & Booth, M. J. Optimum deformable mirror modes for sensorless adaptive optics. Optics Communications (2009)

2009
[11]

& La Riviere, P

Reiser, N., Guo, M., Shroff, H. & La Riviere, P. J. Phase diverse phase retrieval for microscopy: Comparison of gaussian and poisson approaches. arXiv (2023)

2023
[12]

Royer, L. A. et al. Adaptive light-sheet microscopy for long-term, high-resolution imaging in living organisms. Nature Biotechnology (2016)

2016
[13]

van der Walt, S. et al. scikit-image: image processing in Python. PeerJ (2014)

2014
[14]

R., Chan, T

Vogel, C. R., Chan, T. F. & Plemmons, R. J. Fast algorithms for phase-diversity-based blind deconvolution. Adaptive Optical System Technologies (1998)

1998
[15]

& Kanazawa, A

Fridovich-Keil, S., Meanti, G., Warburg, F., Recht, B. & Kanazawa, A. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. arXiv (2023)

2023
[16]

& Sun, J

He, K., Zhang, X., Ren, S. & Sun, J. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015). 18 Supplementary Information Fig. 1 Comparison of runtimes and corresponding achieved DCT norm of the object estimate for the Gaussian, Pois...

2015

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Hampson, K. M. et al. Adaptive optics for high-resolution imaging. Nature Reviews Methods Primers (2021)

work page 2021

[2] [2]

Poyneer, L. A. Scene-based shack-hartmann wave-front sensing: analysis and simulation. Appl. Opt. (2003)

work page 2003

[3] [3]

Kang, I., Zhang, Q., Yu, S. X. & Ji, N. Coordinate-based neural representations for computational adaptive optics in widefield microscopy. Nature Machine Intelligence (2024)

work page 2024

[4] [4]

Guo, M. et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. Nature Communications (2025)

work page 2025

[5] [5]

Wijethilake, N. et al. DEEP-squared: deep learning powered De-scattering with Excitation Patterning. Light: Science & Applications (2023)

work page 2023

[6] [6]

Feng, B. Y. et al. NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media. Science Advances (2023)

work page 2023

[7] [7]

Johnson, C. et al. Phase-diversity-based wavefront sensing for fluorescence microscopy. Optica (2024)

work page 2024

[8] [8]

& Jin, Z

Liu, D., Liu, H. & Jin, Z. Direct phase sensing via an end-to-end model under high-order aberrations. Journal of the Optical Society of America A (2025)

work page 2025