The reviewed record of science sign in
Pith

arxiv: 2606.21381 · v1 · pith:43IXDYQQ · submitted 2026-06-19 · cs.CV · cs.GR· physics.optics

OSOG: A Differentiable, Physics-Informed Synthetic Data Engine for Micro-Optical Environments

Caio Silva This is my paper

Reviewed by Pith2026-06-26 14:29 UTCgrok-4.3pith:43IXDYQQopen to challenge →

classification cs.CV cs.GRphysics.optics
keywords synthetic data generationcomputational microscopydifferentiable renderingobject detectionwave opticsphase retardationzero-shot transfermicro-optical environments
0
0 comments X

The pith

A physics-based synthetic engine produces microscope images that train detectors to work on real occluded samples without fine-tuning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces OSOG as a fully differentiable engine that renders synthetic micro-optical scenes from established models of diffraction and phase retardation inside a PyTorch Structure-of-Arrays layout. It shows that detectors trained exclusively on this synthetic output transfer directly to real Lysozyme micrographs containing heavy occlusion. The same engine supports inverse recovery of optical parameters and runs at speeds that permit on-the-fly dataset creation. If the distribution match holds, the approach removes the annotation bottleneck that currently limits deep learning in computational microscopy.

Core claim

OSOG maps continuous Optical Path Difference calculations into an optimized PyTorch-native architecture that synthesizes 40,000 complex wave-optic particles in under 50 milliseconds while remaining end-to-end differentiable; detectors trained only on the resulting images achieve robust zero-shot transfer to real highly occluded Lysozyme micrographs, and the differentiability further permits exact recovery of continuous optical parameters through curriculum-guided inverse rendering.

What carries the argument

OSOG, the high-performance fully differentiable forward-modeling engine that converts diffraction and phase-retardation models into a PyTorch Structure-of-Arrays tensor pipeline.

If this is right

  • YOLOv11-OBB models trained purely on OSOG data achieve robust zero-shot transfer to real-world highly occluded Lysozyme micrographs.
  • End-to-end differentiability enables exact recovery of continuous optical parameters via curriculum-guided inverse rendering in DiffOSOG.
  • The engine bypasses O(N) sequential ray-tracing bottlenecks and synthesizes 40,000 particles in under 50 milliseconds at over 20 FPS.
  • The tensor pipeline supports true real-time, on-the-fly dataset generation for deep learning in micro-optical settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same forward model could be applied to other dense particle microscopy modalities once their diffraction parameters are supplied.
  • Curriculum-guided inverse rendering might be used to calibrate unknown microscope settings from a small set of real images.
  • Real-time synthesis opens the possibility of generating fresh training batches during detector optimization rather than pre-generating fixed datasets.

Load-bearing premise

The images produced by the diffraction and phase-retardation models inside OSOG have statistics close enough to real Lysozyme micrographs that zero-shot transfer works without domain adaptation.

What would settle it

Train YOLOv11-OBB on OSOG data alone, then evaluate mean average precision on a held-out collection of real Lysozyme micrographs; performance near or above real-data baselines supports the claim while a sharp drop refutes it.

Figures

Figures reproduced from arXiv: 2606.21381 by Caio Silva.

Figure 1
Figure 1. Figure 1: FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
read the original abstract

Deep learning in computational microscopy is severely constrained by the scarcity of densely annotated datasets. While synthetic data generation has bridged this gap in macroscopic computer vision, traditional graphics engines rely on geometric ray-tracing, failing to capture the micro-optical phenomena required for microscopy. Conversely, while wave-optics formulations exist, rendering them computationally tractable at the scale required for deep learning remains a massive systems challenge. To address this, we introduce the Optical Synthetic Object Generator (OSOG), a high-performance, fully differentiable forward-modeling engine. Drawing on established physical models of diffraction and phase retardation, OSOG maps continuous Optical Path Difference (OPD) calculations into a highly optimized, PyTorch-native Structure-of-Arrays (SoA) architecture. We validate this computational framework across three axes: First, object detection models (YOLOv11-OBB) trained purely on OSOG-generated data achieve robust zero-shot transfer to real-world highly occluded Lysozyme micrographs. Second, we introduce DiffOSOG, demonstrating that the engine's end-to-end differentiability allows for the exact recovery of continuous optical parameters via curriculum-guided inverse rendering. Finally, OSOG bypasses the $\mathcal{O}(N)$ bottlenecks of sequential ray-tracing, demonstrating sub-linear scaling by synthesizing 40,000 complex wave-optic particles in under 50 milliseconds (\>20 FPS). By providing a fast, scalable, and physically grounded tensor pipeline, OSOG enables true real-time, on-the-fly dataset generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces the Optical Synthetic Object Generator (OSOG), a high-performance, fully differentiable PyTorch-native engine for synthetic data generation in micro-optical settings. It maps continuous Optical Path Difference (OPD) calculations using established models of diffraction and phase retardation into a Structure-of-Arrays architecture. The central claims are that YOLOv11-OBB detectors trained purely on OSOG data achieve robust zero-shot transfer to real highly occluded Lysozyme micrographs, that end-to-end differentiability enables DiffOSOG for curriculum-guided inverse rendering to recover optical parameters, and that the engine scales sub-linearly to synthesize 40,000 complex particles in under 50 ms (>20 FPS).

Significance. If the zero-shot transfer result is robust and attributable to the wave-optics components rather than generic data variation, OSOG would provide a valuable scalable tool for addressing annotated data scarcity in computational microscopy, enabling on-the-fly generation of physically grounded training sets without reliance on real microscope time.

major comments (2)
  1. [Abstract] Abstract, first validation axis: the claim of 'robust zero-shot transfer' to real-world Lysozyme micrographs is presented without any quantitative metrics (mAP, precision-recall, error bars), baseline comparisons, or details on the real test set (size, acquisition parameters such as NA/wavelength), preventing assessment of whether the result supports the central justification for the physics-informed engine.
  2. [Abstract] Abstract, first validation axis: no ablation is reported that replaces the diffraction/phase-retardation forward model with a geometric ray-tracing baseline or simple augmentation while holding other factors (label density, occlusion statistics, volume) fixed; without this, it is impossible to determine if the specific wave-optics components are load-bearing for transfer or if success arises from orthogonal properties of the synthetic distribution.
minor comments (1)
  1. [Abstract] Abstract: the performance claim of synthesizing 40,000 particles in under 50 ms should specify the hardware platform and whether timing includes the full differentiable pipeline or only the forward rendering step.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which highlight opportunities to strengthen the abstract's self-contained presentation of our validation results. We will revise the abstract accordingly while preserving the manuscript's core contributions.

read point-by-point responses

  1. Referee: [Abstract] Abstract, first validation axis: the claim of 'robust zero-shot transfer' to real-world Lysozyme micrographs is presented without any quantitative metrics (mAP, precision-recall, error bars), baseline comparisons, or details on the real test set (size, acquisition parameters such as NA/wavelength), preventing assessment of whether the result supports the central justification for the physics-informed engine.

    Authors: We agree that the abstract would be strengthened by including quantitative metrics and test-set details to support the zero-shot transfer claim. In the revised manuscript we will update the abstract to report the specific mAP@0.5 and mAP@0.5:0.95 values (with standard deviations across runs), precision-recall characteristics, and real test-set information including image count and acquisition parameters (NA and wavelength). These quantities already appear in Section 4 and Table 1; embedding the key figures in the abstract will make the central claim directly verifiable without requiring the reader to consult the body text. revision: yes

  2. Referee: [Abstract] Abstract, first validation axis: no ablation is reported that replaces the diffraction/phase-retardation forward model with a geometric ray-tracing baseline or simple augmentation while holding other factors (label density, occlusion statistics, volume) fixed; without this, it is impossible to determine if the specific wave-optics components are load-bearing for transfer or if success arises from orthogonal properties of the synthetic distribution.

    Authors: This observation is correct: the current abstract (and main text) does not present a controlled ablation that isolates the wave-optics forward model against a matched geometric baseline. We will add such an ablation to the revised manuscript by generating a geometric-optics variant of OSOG under identical label density, occlusion, and volume statistics, retraining the detector, and quantifying the resulting drop in zero-shot transfer performance. The outcome will be summarized in the revised abstract to demonstrate that the diffraction and phase-retardation components are load-bearing. revision: yes

Circularity Check

0 steps flagged

No circularity; physics models external and results empirical

full rationale

The paper presents OSOG as implementing established external physical models of diffraction and phase retardation (abstract: 'Drawing on established physical models of diffraction and phase retardation'), with the zero-shot transfer result being an empirical validation on real Lysozyme micrographs rather than a derived claim that reduces to the model's own fitted parameters or self-citations. No equations, self-citations, or ansatzes are shown that would make any prediction equivalent to its inputs by construction. The differentiability and scaling claims are implementation and benchmarking results, not load-bearing derivations that collapse into tautology. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are detailed beyond the high-level claim of using established diffraction and phase-retardation models.

axioms (1)
  • domain assumption Established physical models of diffraction and phase retardation accurately capture micro-optical phenomena in the target domain
    Invoked to justify the forward-modeling engine

pith-pipeline@v0.9.1-grok · 5797 in / 993 out tokens · 17879 ms · 2026-06-26T14:29:38.044175+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, Domain randomization for transferring deep neural networks from simulation to the real world, in 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS)(IEEE, 2017) pp. 23–30

    2017

  2. [2]

    Tremblay, A

    J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jam- pani, C. Anil, T. Foley, S. Garza, L. Isaacs, H. Snipt, et al., Training deep networks with synthetic data: Bridg- ing the reality gap by domain randomization, inProceed- ings of the IEEE conference on computer vision and pat- tern recognition workshops(2018) pp. 969–977

    2018

  3. [3]

    H. Kato, Y. Ushiku, and T. Harada, Neural 3d mesh ren- derer, inProceedings of the IEEE conference on computer vision and pattern recognition(2018) pp. 3907–3916

    2018

  4. [4]

    Mildenhall, P

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, Nerf: Representing scenes as neural radiance fields for view synthesis, inEuropean conference on computer vision(Springer, 2020) pp. 405– 421

    2020

  5. [5]

    Bouchama, B

    L. Bouchama, B. Dorizzi, J. Klossa, and Y. Gottesman, Deep learning framework for an efficient fourier ptycho- graphic microscopy reconstruction under low overlap con- ditions, Sensors23, 6829 (2023)

    2023

  6. [6]

    R. Sun, D. Yang, S. Zhang, and Q. Hao, Hy- brid deep learning and physics-based neural net- work for programmable illumination computa- tional microscopy, arXiv preprint arXiv:2403.12970 10.48550/arxiv.2403.12970 (2024)

    work page doi:10.48550/arxiv.2403.12970 2024

  7. [7]

    Deb, G.-J

    D. Deb, G.-J. Both, E. Bezzam, A. Kohli, S. Yang, A. Chaware,et al., Chromatix: a differentiable, gpu- accelerated wave-optics library, bioRxiv , 2025 (2024)

    2025

  8. [8]

    Ichbiah, A

    S. Ichbiah, A. Sinha, F. Delbary, and H. Turlier, Inverse 3d microscopy rendering for cell shape inference with ac- tive mesh, inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)(2025)

    2025

  9. [9]

    Morgado, E

    L. Morgado, E. G´ omez-de Mariscal, H. S. Heil, and R. Henriques, The rise of data-driven microscopy pow- ered by machine learning, Journal of Microscopy295, 85 (2024)

    2024

  10. [10]

    D. S. Ebert, F. K. Musgrave, D. Peachey, K. Perlin, and S. Worley,Texturing and modeling: a procedural approach (Morgan Kaufmann, 2003)

    2003

  11. [11]

    R. M. Smelik, T. Tutenel, R. Bidarra, and B. Benes, A survey on procedural modelling for virtual worlds, Com- puter Graphics Forum33, 31 (2014). 9

    2014

  12. [12]

    Pharr, W

    M. Pharr, W. Jakob, and G. Humphreys,Physically based rendering: From theory to implementation(Mor- gan Kaufmann, 2016)

    2016

  13. [13]

    Walter, S

    B. Walter, S. R. Marschner, H. Li, and K. E. Torrance, Microfacet models for refraction through rough surfaces, inProceedings of the 18th Eurographics conference on Rendering Techniques(2007) pp. 195–206

    2007

  14. [14]

    L. A. Gatys, A. S. Ecker, and M. Bethge, A neural algo- rithm of artistic style, arXiv preprint arXiv:1508.06576 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015