{"id":"3a6c39ae-5420-4ccb-bd35-c818ceda4ebb","arxiv_id":"2507.07789","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"IDEAL-IO decouples density estimation from optical optimization to make information-theoretic imaging design practical, cutting runtime and memory by up to 6x while enabling more expressive density models.","lead":"This paper introduces IDEAL-IO, a faster and more memory-efficient way to design imaging systems by maximizing how much information their measurements capture. It alternates between fitting a statistical model of the measurements and updating the optical elements, cutting compute by up to 6x while matching or improving design quality in simulations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gradient bias from dropping ∂ψ/∂θ is unquantified; the real condition is whether ψ is at the fitted optimum, not whether ∂ψ/∂θ is small.","rationale":"The reader identified the omitted ∂ψ/∂θ term, and I agree this is the key spot; however, the precise failure condition is different. Under exact maximum-likelihood fitting, the term is zero by the envelope theorem, so the unquantified empirical claim that ∂ψ/∂θ is small is not the right justification. With PixelCNN refit only every K steps, and with a possible train/test split between fitting and evaluation, the fitted ψ is not the argmin of the evaluated loss; the dropped term is then nonzero and unmeasured. No experiment in the paper checks this directly. A finite-difference gradient comparison with the Gaussian model would settle it. This is an addressable concern, not a demonstrated failure; the E2E comparison in Sec. 4.1 provides some supporting evidence that the biased objective still correlates with design quality. Therefore I keep the reader's CONDITIONAL verdict: the paper should quantify the omitted term before the central quality claim is accepted.","tokens_in":8892,"tokens_out":8245,"duration_ms":96717,"concrete_test":"Reproduce the smallest DOE experiment (4×4 patches) with the analytic Gaussian density model. At 10 points on the optimization trajectory compute (a) the IDEAL-IO gradient ∂L/∂θ with ψ frozen, and (b) the total derivative dL/dθ = ∂L/∂θ + ∂L/∂ψ dψ/dθ, where dψ/dθ is obtained by differentiating the analytic maximum-likelihood fit or by central finite differences in θ with ψ re-fit for each perturbed θ. Report cosine similarity and relative norm error between (a) and (b), plus max ||∂ψ/∂θ||/||ψ||. If cosine < 0.9 or relative error > 20% at any checkpoint, the Eq. 8 gradient is materially biased and the conditional verdict should be tightened; if they agree, replace the '∂ψ/∂θ is small' justification with the envelope-theorem argument.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing risk is not the magnitude of ∂ψ/∂θ itself but whether the refit density model is at an optimum of the loss being differentiated. In the full total derivative (Eq. 6), the term ∂p/∂ψ ∂ψ/∂θ is dropped in Eq. 8. If ψ_{t+1} exactly minimizes the cross-entropy on the same measurements used for the gradient, then ∂L/∂ψ = 0 by first-order optimality, so the omitted term vanishes regardless of ∂ψ/∂θ (envelope theorem). The paper instead asserts that ∂ψ/∂θ is small and gives no measurement of either ∂ψ/∂θ or ∂L/∂ψ. In the actual schedule, PixelCNN is refit only every K=50 optical updates (Sec. 4.3), and if held-out measurements are used for entropy estimation (Eqs. 3–4), ψ is not the minimizer of the evaluated loss. Then the dropped term is nonzero, Eq. 8 is a biased gradient of the cross-entropy/MI objective, and convergence to a design that maximizes true MI is unsupported. The efficiency claims survive, but the 'maintaining or improving design quality' claim depends on this bias being small.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IDEAL-IO, an alternating optimization scheme for information-driven optical design. Instead of differentiating through the density-model fitting procedure as in the original IDEAL, IDEAL-IO alternates between fitting a density model to measurements from the current optical system and updating the optical parameters while holding the fitted density model fixed. The central approximation is that the term ∂ψ/∂θ in the total derivative of the mutual-information estimate is small, so the gradient update in Eq. (8) ignores it. The authors report up to 6× reductions in runtime and memory, the ability to use expressive density models such as PixelCNN, and validation on snapshot 3D microscopy, diffractive optical elements, and lensless imaging.","tokens_in":9193,"tokens_out":5509,"duration_ms":61164,"significance":"If the claims hold, IDEAL-IO makes information-theoretic optical design substantially more practical by removing the differentiability requirement on density estimation. The paper's strengths include measured memory and runtime scaling (Fig. 3b–c, Table 1), an independent reconstruction-fidelity comparison in Sec. 4.1 using NMSE from a separately trained decoder, and the observation that the DOE optimization converges to a known Fresnel zone plate. These elements provide credible evidence for the efficiency contribution. However, the central gradient approximation is not quantified, and the lensless 'superior design' claim in Sec. 4.3 is evaluated with the same PixelCNN mutual-information quantity that guides the optimization, which is circular. The efficiency result is sound, but the design-quality claim needs additional support before the paper's full message can be accepted.","major_comments":[{"comment":"The central approximation ∂ψ/∂θ≈0 is stated as an empirical observation but never quantified. More importantly, the correct sufficient condition for the dropped term to vanish is not smallness of ∂ψ/∂θ but first-order optimality of ψ with respect to the loss being differentiated: if ψ_{t+1} exactly minimized the cross-entropy on the same measurements used in the gradient, the term ∂p/∂ψ ∂ψ/∂θ would vanish by the envelope theorem. The paper's actual schedule violates this condition: the PixelCNN is refit only every K=50 optical updates (Sec. 4.3), and the entropy estimate uses held-out measurements (Eqs. 3–4), so ψ is not at the optimum of the evaluated loss. The authors should provide a direct measurement of the dropped gradient term or of ∂L/∂ψ over training, or demonstrate that final designs are insensitive to K and to the approximation. Without this, the claim that IDEAL-IO maintains or improves design quality while maximizing true mutual information is unsupported.","section":"Section 3, Eq. (8)"},{"comment":"The lensless imaging evaluation uses PixelCNN mutual information on held-out test data as the design-quality metric, which is exactly the quantity that the PixelCNN-guided optimization maximizes. The finding that PixelCNN-optimized designs achieve higher PixelCNN MI is therefore expected and does not independently establish that the designs are 'superior.' The manuscript needs an external evaluation metric, such as reconstruction NMSE with a separately trained decoder (as done in Sec. 4.1) or a downstream task accuracy, to support the abstract's claim that expressive models 'guide optimization toward superior designs.' Without such an independent metric, the lensless result only demonstrates that the optimizer succeeds at maximizing its own surrogate.","section":"Section 4.3, Fig. 4c"}],"minor_comments":[{"comment":"The abstract claims 'up to 6×' reduction, but the reported results are approximately 4× faster than the E2E baseline, roughly 3× faster than IDEAL, and about 5.6× lower peak memory; please specify which quantity yields the 6× factor or adjust the claim.","section":"Abstract and Sec. 4.2"},{"comment":"It is unclear whether ψ is fit on the same measurements used for the entropy estimate in Eq. (8); if the training and evaluation sets differ, the envelope-theorem reasoning is unavailable and the bias from the dropped term should be discussed explicitly.","section":"Section 3, Eqs. (3)–(4) and (7)–(8)"},{"comment":"The text states that IDEAL-IO's training curve appears noisier due to its smaller test set, but it also states that both methods used 6480 patches; please clarify the test-set sizes used for the loss curves.","section":"Section 4.2"},{"comment":"The figure label says 'Training decreases error and increases information,' but the plot shows MI versus NMSE at three checkpoints, not training curves; please relabel or replot as functions of training step.","section":"Section 4.1, Fig. 2c"},{"comment":"The word 'fluorescense' should be 'fluorescence.'","section":"Section 4.1"},{"comment":"References [7] and [9] appear to describe the same work, one listed as an arXiv preprint and one as anonymous under review; please reconcile these to avoid duplicate self-citation.","section":"References"},{"comment":"Final MI values for CIFAR10 and MNIST are reported without standard deviations or the number of random seeds; please add error bars or at least state the number of independent runs.","section":"Section 4.3"},{"comment":"Several derivations and model details are deferred to the Supplement (Secs. 2, 7, 9), but the supplement is not included with the manuscript; please ensure it is available to reviewers.","section":"Supplement"}],"recommendation":"major_revision","confidential_remarks":"The efficiency contribution is well supported by the scaling experiments and should be preserved. The main risk is that the design-quality claims in Sec. 4.3 rest on a circular metric and the core gradient approximation is unquantified. I would encourage the editor to require either a direct quantification of the dropped gradient term or an independent evaluation of the lensless designs before acceptance. The paper is likely to be acceptable after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Key thing to know: this is a pragmatic extension of IDEAL that alternates density fitting and optical updates, with believable efficiency gains, but the central approximation is under-justified. I would treat it as conditional, not a reject.\n\nWhat is new and good: The alternating update (Eqs. 7-8) is a coordinate-descent-style decoupling: fit pψ to current measurements, then update θ with ψ frozen. That lets them replace Gaussian density models with PixelCNN, which is a real first for MI-based optical design. The efficiency claims are strong: measured memory and runtime scaling (Fig. 3b-c, Table 1) show roughly 6x improvement, and the DOE task with 6.2M parameters runs in ~120s vs ~360s and 7.9 vs 44 GB. They also did useful independent checks: 3D microscopy compares against a separately trained decoder via NMSE, and DOE optimization converges to a Fresnel zone plate, which is the known optimal design. That is good evidence the method is not just optimizing a meaningless objective.\n\nSoft spots: The load-bearing assertion is that ∂ψ/∂θ is small, stated in Section 3 as an empirical observation, never measured or bounded. The stress-test point is sharper: the right condition is whether ψ is at an optimum of the loss being differentiated. If ψ_{t+1} exactly minimized the cross-entropy on the same measurements used for the gradient, the omitted ∂L/∂ψ term vanishes by first-order optimality, regardless of ∂ψ/∂θ. But they refit PixelCNN only every K=50 optical steps (Sec 4.3) and use held-out patches for entropy estimation, so ψ will not generally be at the optimum of the evaluated loss. Then Eq. 8 is a biased gradient of the MI/CE objective, and the claim that IDEAL-IO maintains or improves design quality is unsupported until the bias is quantified or a convergence analysis is provided. The efficiency evidence survives; the theoretical framing does not.\n\nMinor issues: The lensless 'better design' claim (Sec 4.3) is evaluated by MI estimated with PixelCNN—the same quantity being optimized—so it is partly circular. The CIFAR10 result is roughly flat, which is consistent with near-Gaussian measurements. The 3D microscopy comparison uses only a few checkpoints and no error bars, though the trend is plausible. No code release is mentioned, which matters for a methods paper. Citation to the group's own unpublished work is heavy but not inappropriate in this small field.\n\nBottom line: This deserves serious peer review and would benefit the computational imaging community. A careful referee should ask for quantification of the omitted term or a proof-style argument under stated conditions, plus task-level benchmarks. That is the difference between a solid workshop paper and a solid journal paper.","headline":"A useful empirical extension of IDEAL with credible efficiency gains, but the unquantified ∂ψ/∂θ approximation keeps the design-quality claims conditional.","tokens_in":9690,"tokens_out":3422,"would_cite":true,"duration_ms":39028,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Information-driven optical design can be made up to 6x cheaper by alternating density fitting and optical updates.","keywords":["computational imaging","information-driven optical design","mutual information","alternating optimization","density estimation","diffractive optics","lensless imaging","snapshot 3D microscopy"],"falsifier":"Compute the full gradient including $\\partial\\psi/\\partial\\theta$ whenever the density model fitting is differentiable, and compare the resulting optical update trajectories with the IDEAL-IO approximation on the same tasks and initialization; if the final designs or the mutual-information values diverge substantially, particularly on non-Gaussian benchmarks such as MNIST, the small-gradient assumption is violated.","tokens_in":1722,"feed_emoji":"🔭","tokens_out":1967,"duration_ms":63038,"temperature":0.7,"pith_summary":"Information-driven optical design aims to choose a lens, phase mask, or other optical element by maximizing the mutual information between scenes and measurements, without training a reconstruction network. This paper argues that the standard way of performing that optimization, which backpropagates through the density model used to estimate entropy, carries high memory and runtime costs and restricts the choice of density models to simple, differentiable ones. The proposed method, IDEAL-IO, alternates between fitting the density model to current measurements with gradients frozen and updating the optical parameters with that model held fixed. This removes the need to differentiate through the fitting process, cutting runtime and memory usage by up to 6x while permitting expressive models such as PixelCNN. The paper validates the approach on snapshot 3D microscopy, diffractive optical elements, and lensless imaging, and reports that the cheaper optimization matches or improves design quality.","feed_headline":"Decoupling density fitting makes optical design up to 6x faster","feed_subtitle":"Swapping to fit-then-optimize keeps mutual-information lens design accurate while slashing memory and runtime.","key_machinery":"The objective is the mutual information $\\hat{I}(X;Y) = \\hat{H}(Y) - H(Y|X)$, where $H(Y|X)$ is computed analytically from the noise model and $\\hat{H}(Y)$ is a cross-entropy estimate on held-out noisy measurements using a fitted density model $p_\\psi$. The carrying mechanism is the alternating scheme: fit $\\psi$ to measurements generated with the current optical parameters, then update the optical parameters using the fixed density model and the known noise law. The validity of the scheme rests on omitting the term $\\partial p/\\partial\\psi \\cdot \\partial\\psi/\\partial\\theta$ from the optical gradient, which the paper justifies by an empirical claim that the fitted distribution parameters change slowly with respect to the imaging-system parameters. Patch-based entropy estimation keeps the memory footprint manageable and is the setting in which the 6x runtime and memory reductions are measured.","core_discovery":"The central claim is that the gradient of the fitted density-model parameters with respect to the optical parameters, $\\partial\\psi/\\partial\\theta$, is small enough to omit from the mutual-information gradient, and that omitting it converts the costly monolithic backpropagation of IDEAL into two cheap alternating steps. Concretely, the optical update in Equation 8 drops the term $\\frac{\\partial p}{\\partial\\psi}\\frac{\\partial\\psi}{\\partial\\theta}$ while the density model is refit outside the differentiation path as in Equation 7. The paper reports that this reduces runtime and GPU memory by up to 6x compared with original IDEAL, and it enables density models that are not differentiable or not practical to differentiate, such as PixelCNN. On lensless imaging with highly non-Gaussian measurements, the PixelCNN-guided design achieves substantially higher test mutual information than a Gaussian-guided design, indicating that the decoupling does more than save compute: it changes which designs are reachable.","pith_inferences":["Editorial inference: the smallness of $\\partial\\psi/\\partial\\theta$ is likely to depend on how far the optimization is from convergence and on the density model family; if it becomes nonnegligible early in training, the effective objective could differ from true mutual information in ways that the reported loss curves would only partially reveal.","Editorial inference: the decoupling principle could extend beyond mutual information to other information-theoretic objectives, such as task-specific or class-conditional information, by alternating any differentiable information estimator with any nondifferentiable fitting procedure.","Editorial inference: because IDEAL-IO is cheap enough to run many initializations, it could serve as a fast warm-start for end-to-end design, first exploring the optical parameter space with information maximization and then fine-tuning jointly with a decoder.","Editorial inference: the patch-size trade-off the paper identifies suggests a natural testable extension toward multiscale or hierarchical density models, which could capture larger spatial correlations without the memory cost of full-frame estimation."],"forward_implications":["Scaling improves substantially: for the diffractive-optics task, IDEAL-IO required about 7.9 GB of GPU memory where IDEAL required 44.2 GB, and its runtime per optimization step grew roughly 0.007 seconds per 100 additional patch pixels versus roughly 0.098 seconds for IDEAL.","Expressive density models become usable, and they change designs: on MNIST lensless imaging, the PixelCNN-optimized design reached 0.473 bits/pixel of test mutual information versus 0.208 for the Gaussian-optimized design.","Mutual-information maximization can replace end-to-end training as a practical objective: in snapshot 3D microscopy, IDEAL-IO converged in about 25 minutes without training a decoder, about 4x faster than the roughly 2-hour end-to-end baseline, while producing a similar lenslet design.","The method does not sacrifice design quality on the tested tasks; for the diffractive element, both IDEAL and IDEAL-IO converged to Fresnel-like zone-plate designs without explicit structural priors.","Because the fitting and optimization stages are decoupled, the same framework can be applied to other density model families and other differentiable optical forward models without re-deriving end-to-end gradients."],"supporting_citations":[{"why":"Supplies the original IDEAL formulation and its end-to-end differentiability requirement, which this paper identifies as the bottleneck and removes.","marker":"[9]"},{"why":"Provides the PixelCNN autoregressive density model used as the expressive non-Gaussian estimator enabled by the decoupling.","marker":"[10]"},{"why":"Supplies the end-to-end diffractive optical element optimization code and design task that IDEAL-IO reproduces at lower computational cost.","marker":"[3]"},{"why":"Provides the end-to-end Fourier DiffuserScope design baseline for snapshot 3D microscopy that IDEAL-IO matches with roughly 4x less runtime.","marker":"[14]"},{"why":"Defines the snapshot 3D Fourier light field microscopy architecture used as a testbed in the microscopy comparison.","marker":"[12]"},{"why":"Motivates the lensless imaging comparison by arguing that maximizing information capture improves lensless system design.","marker":"[17]"}],"fun_headline_variants":["6x faster optical design via alternating optimization","Alternating optimization cuts optical design cost 6x","Fit then optimize: 6x speedup for optical design","Decoupled density fitting speeds optical design 6x","IDEAL-IO: 6x faster lens design with better models"],"cache_read_input_tokens":11904,"weakest_assumption_plain":"The method assumes that the fitted density-model parameters change only slowly as the optical parameters change, so ignoring the gradient of those parameters in the optical update does not bias the optimization; the paper states this as an empirical observation but does not quantify or bound the error.","fun_headline_variants_meta":{"raw":{"variants":["6x faster optical design via alternating optimization","Alternating optimization cuts optical design cost 6x","Fit then optimize: 6x speedup for optical design","Decoupled density fitting speeds optical design 6x","IDEAL-IO: 6x faster lens design with better models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1339,"prompt_tokens":915,"completion_tokens":424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":344}},"tokens_in":531,"tokens_out":424,"duration_ms":4499,"temperature":1.0,"reasoning_tokens":344,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:31:18.233480+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the full gradient including $\\partial\\psi/\\partial\\theta$ whenever the density model fitting is differentiable, and compare the resulting optical update trajectories with the IDEAL-IO approximation on the same tasks and initialization; if the final designs or the mutual-information values diverge substantially, particularly on non-Gaussian benchmarks such as MNIST, the small-gradient assumption is violated.","supporting_citations":[{"cited_title":"Information-driven design of imaging systems","cited_arxiv_id":null,"evidence_quote":"Supplies the original IDEAL formulation and its end-to-end differentiability requirement, which this paper identifies as the bottleneck and removes."},{"cited_title":"End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging","cited_arxiv_id":null,"evidence_quote":"Supplies the end-to-end diffractive optical element optimization code and design task that IDEAL-IO reproduces at lower computational cost."},{"cited_title":"Physics-based learned design for fourier diffuserscope","cited_arxiv_id":null,"evidence_quote":"Provides the end-to-end Fourier DiffuserScope design baseline for snapshot 3D microscopy that IDEAL-IO matches with roughly 4x less runtime."},{"cited_title":"Fourier DiffuserScope: Single-shot 3D Fourier light field microscopy with a diffuser","cited_arxiv_id":"2006.16343","evidence_quote":"Defines the snapshot 3D Fourier light field microscopy architecture used as a testbed in the microscopy comparison."},{"cited_title":"Designing lensless imaging systems to maximize information capture","cited_arxiv_id":null,"evidence_quote":"Motivates the lensless imaging comparison by arguing that maximizing information capture improves lensless system design."}],"review_version":1}