{"id":"38228b06-da85-4d69-99a6-5d35b65da85f","arxiv_id":"2412.16028","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CoCoGaussian reconstructs sharp 3D scenes from defocused multi-view photos by modeling the circle of confusion with extra 3D Gaussians.","lead":"CoCoGaussian trains a 3D Gaussian Splatting model on defocused photos only and reconstructs the sharp 3D scene by modeling each blur disk as a circle of confusion. It reports state-of-the-art rendering quality on defocus benchmarks and adds post-capture control of aperture and focus plane.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Euclidean depth in Eq. (7) makes the focus 'plane' a sphere around the camera; off-axis CoC sizes are miscalibrated and β≤1 cannot fix underestimates.","rationale":"The reader identified the Euclidean-distance depth proxy as the weakest assumption; I agree that this is the most load-bearing technical point, and I sharpen it in two ways. First, the approximation is not merely an overestimate of depth: when a Gaussian lies in front of the focus surface, the radial-distance rounding makes |1/d - 1/dF| smaller than the true axial value, so the predicted CoC is underestimated; since β ≤ 1, the model cannot enlarge it. Second, the 'focus plane' is geometrically a sphere, not a plane, so the physical prior is miscalibrated across the image in a way that a single global K cannot correct. The empirical SOTA results could still hold because the learnable β, direction vectors, and CNN weight map absorb the error, but then the contribution of the physical CoC prior to the sharp output is less than claimed. The paper's own ablation 'w/o aperture parameter' shows that removing K leaves a fairly strong baseline, which reinforces that learned flexibility, rather than the physical formula, does much of the work. I therefore think the claim 'physically grounded' is overstated, and the sharp-reconstruction claim should be conditioned on a validation of the depth model. The proposed re-training experiment with axial depth would directly quantify whether the Euclidean choice degrades the CoC model. Since the paper otherwise presents a well-engineered method with strong results and appropriate ablations, I do not see a reason to reject; CONDITIONAL remains the right verdict.","tokens_in":20018,"tokens_out":13585,"duration_ms":129098,"concrete_test":"Retrain CoCoGaussian on the DoF-NeRF real-world dataset with the depth in Eq. (7) replaced by the axial depth d(µ_B) = (µ_B - x_cam)·v, where v is the unit optical-axis vector from the camera pose, keeping dF as the axial focus distance and all other components (MLP, CNN, losses, M=5) identical. Compare LPIPS/PSNR/SSIM against the published Euclidean-depth version (Table 2). If the axial-depth variant achieves comparable or better LPIPS (within 0.005), the Euclidean assumption is not load-bearing for the empirical claim; if it improves by more than 0.01 LPIPS or visibly sharpens off-axis regions, the current radial-depth choice is a measurable source of bias. Additionally, on a wide-FOV synthetic scene, render defocused images with a planar focus surface and compare the predicted CoC diameters from Eq. (7) against the ground-truth axial-depth values to quantify the angular error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core physical prior in Eq. (7) computes d(µ_B) as the Euclidean distance from the camera center x_cam to the Gaussian mean µ_B. In a real thin-lens camera, the object distance that enters the CoC formula is the axial distance along the optical axis, not the radial distance. Using radial distance means that the set of points at distance dF is a sphere around x_cam, so the 'focus plane' is actually a spherical surface. For a point at angular offset θ from the optical axis, the radial depth overestimates the axial depth by 1/cos θ. This introduces a spatially varying error in σ(µ_B): for points behind the focus surface the CoC is overestimated, while for points in front of it the CoC is underestimated. Because K is a single scalar learned globally, it cannot remove this angular dependence, and because the scaling factors β_m are constrained to (0,1], the model can only shrink the CoC offsets, never enlarge an underestimated σ. The network could in principle compensate via the learned direction vectors and the CNN weight map, but then the sharpness of the base-Gaussian render is produced by learned corrections rather than by the physical CoC model. This directly weakens the central claim that CoCoGaussian 'accurately models defocus blur by constructing the CoC through 3D Gaussians and learnable aperture parameters.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoCoGaussian, a 3D Gaussian Splatting method that reconstructs a sharp 3D scene from multi-view defocused images. For each base Gaussian, a Circle of Confusion (CoC) diameter is computed from the Euclidean distance between the camera center and the Gaussian mean, a learned aperture scalar K, and a per-image learned focus distance dF (Eq. 7). The method generates M additional 'CoC Gaussian' sets by offsetting the base Gaussian means along learned unit directions, with offsets scaled by the CoC diameter and learnable factors beta_m in (0,1] (Eq. 9). The base and CoC Gaussians are rasterized into M+1 images, which are combined by a CNN-computed pixel-wise weighted sum. At inference, rendering only the base Gaussians is claimed to produce sharp novel views, and K and dF can be adjusted for depth-of-field and focus-plane customization. Experiments on Deblur-NeRF synthetic and real datasets, DoF-NeRF real dataset, and NeRF-LLFF show state-of-the-art or competitive metrics, with ablations demonstrating the contribution of each component.","tokens_in":20368,"tokens_out":6812,"duration_ms":59047,"significance":"If the claims hold, CoCoGaussian is a practically valuable contribution: it extends 3DGS to defocused inputs while retaining real-time rendering, and it provides an interpretable, physically motivated mechanism for depth-of-field control. The paper is strong in its breadth of comparisons, reproduction of baseline codes, and explicit ablations of each proposed module. The central idea of generating CoC-shaped Gaussian copies from a per-Gaussian depth and a learned aperture is novel in the 3DGS literature and outperforms previous blur-agnostic Gaussian methods. The main risk is that the physical depth used in the CoC formula is not the axial depth required by the thin-lens model, which weakens the claim that the blur is 'accurately' modeled by the physical CoC; the empirical gains may partly come from the learned adaptive components rather than the physics.","major_comments":[{"comment":"The depth d(µ_B) in Eq. (7) is the Euclidean distance between the camera center x_cam and the Gaussian mean µ_B. In the thin-lens model underlying Eq. (6), the relevant object distance is the axial distance along the optical axis, not the radial distance. For a point at angular offset θ from the optical axis, radial distance overestimates axial distance by a factor of 1/cos θ, causing the CoC diameter to be systematically overestimated for points behind the focus surface and underestimated for points in front of it. Because the scaling factors β_m are constrained to (0,1] in Eq. (9), the model can shrink overestimated CoCs but cannot enlarge underestimated ones; the limitation discussion in Sec. 6 acknowledges only the over-estimation case. The learned direction vectors and CNN weights could in principle compensate, but then the sharp base-Gaussian output is produced by learned corrections rather than by the physical CoC model. Please either replace d(µ_B) with the axial depth in the camera coordinate frame, or provide a quantitative analysis of the angular bias on the tested datasets showing it is negligible.","section":"Sec. 4.2, Eq. (7); Sec. 4.3, Eq. (9); Sec. 6"}],"minor_comments":[{"comment":"There is a typo in the sentence 'reduce dependence on estmiated depth'; 'estmiated' should be 'estimated'.","section":"Sec. 1"},{"comment":"The text says 'However, theses offsets ∆µCoC presents two potential issues'; 'theses' should be 'these' and the verb should agree.","section":"Sec. 4.3"},{"comment":"The sentence 'as demonstrated in Tab. 5 of the main paper' refers to the NeRF-LLFF results, which are in Table 4 of the main paper; the table number is incorrect.","section":"Appendix Sec. 12"},{"comment":"The reference numbers in the table header are inconsistent with the main text: 'DP-NeRF [18]' should be [11], 'Deblurring 3D-GS [25]' should be [10], and 'BAGS [11]' should be [26].","section":"Appendix Table 9"},{"comment":"In the sentence 'the C ISCO and C ORAL scenes have higher luminance', the scene names are broken by spaces; this appears to be a formatting artifact.","section":"Appendix Sec. 10"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a competent, well-tested extension of DoF-NeRF's circle-of-confusion idea into 3D Gaussian Splatting. The new piece is the CoC diameter computed per Gaussian from depth and a learned aperture scalar, the generation of M auxiliary Gaussian sets to form the blur kernel, and the beta scaling that lets the model shrink an overestimated CoC. That combination works: on the three benchmarks the method beats the current blur-aware GS baselines on LPIPS/SSIM, and the ablations show each component earns its place. Real-time rendering is preserved because at inference you just drop the CoC Gaussians.\n\nThe reader's conditional verdict is about right. The stress-test concern about Euclidean depth is a fair point, and the paper's own limitation section concedes that beta only shrinks, not enlarges. Using radial distance from the camera center instead of axial depth means off-axis points get a CoC that is systematically too large behind focus and too small in front; a single learned K cannot fix the angular dependence. The mitigation is that the learned direction vectors and the CNN weight map can absorb the error, so the physical prior becomes a soft regularizer rather than a precise model. I do not think that is fatal to the method, but it does undercut the phrase 'accurately models defocus blur' in the conclusion.\n\nThe bigger practical weaknesses are the standard ones: no code release, no error bars, and K and dF are fit to the same blur they are used to explain, which is self-supervised fitting rather than a measured camera model. The paper is honest about the data issues in the Deblur-NeRF real-world set, and the per-scene tables give a good picture of where the method wins and loses.\n\nWho should read it: anyone working on blur-robust novel view synthesis or camera-aware 3DGS. It deserves a serious referee: the experiments are broad, the ablations are reasonable, and the idea is directly useful even if the physical grounding is approximate. I would accept it for review with a request for code and a more careful statement about depth.","headline":"Solid engineering that transfers DoF-NeRF's circle-of-confusion model into 3DGS with a useful adaptive scaling; the physical depth model is approximate, but the paper is honest about it and the results are strong.","tokens_in":20924,"tokens_out":2147,"would_cite":true,"duration_ms":19974,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoCoGaussian reconstructs sharp 3D scenes from defocused images by modeling each Gaussian's blur as a circle of confusion whose diameter comes from depth and a learned aperture scalar.","keywords":["defocus blur","circle of confusion","3D Gaussian Splatting","novel view synthesis","depth of field","blind deblurring","deblurring radiance fields","aperture estimation"],"falsifier":"Render a synthetic scene containing a planar target tilted about 45 degrees to the optical axis with a known thin-lens aperture, train CoCoGaussian on the resulting defocused views, and compare the recovered CoC diameters of off-axis Gaussians with the ground-truth lens equation: a systematic mismatch that grows with radial distance would confirm that the Euclidean-depth proxy, rather than the learned parameters, is driving the error.","tokens_in":19798,"feed_emoji":"📷","tokens_out":8971,"duration_ms":67411,"temperature":0.7,"pith_summary":"CoCoGaussian claims that defocus blur, usually considered an obstacle to 3D reconstruction, can itself serve as the training signal. For each 3D Gaussian the paper computes a circle-of-confusion diameter from that Gaussian's depth and a learned aperture scalar, then spawns additional Gaussians that reproduce the blur pattern during training. Given only defocused multi-view images, the model renders the blurry training views and, at inference, drops the blur Gaussians to render sharp novel views. If the claim holds, all-in-focus captures are no longer required for 3D scene reconstruction, and the same trained model can refocus or change depth of field after capture.","feed_headline":"Circle-of-confusion model turns defocused photos into sharp 3D views","feed_subtitle":"A learned aperture and per-Gaussian depth explain blur, then render sharp views with adjustable depth of field.","key_machinery":"The central machinery is the generation of CoC Gaussian sets: a small MLP $h_\\theta$ maps the camera position and base Gaussian parameters to the aperture scalar $K$, M direction vectors $d_m$, M scaling factors $\\beta_m \\in (0,1]$, and scale/quaternion deltas. The load-bearing identity is $\\sigma(\\mu_B) \\approx K |1/d(\\mu_B) - 1/d_F|$, which ties each Gaussian's depth to the diameter of its blur disk, and the offset rule $\\mu_{\\text{CoC};m} = \\mu_B + (\\sigma(\\mu_B)/2)\\,\\beta_m d_m$, which places the spawned Gaussians inside that disk. This is what lets the model render defocus during training and simply omit it at inference.","core_discovery":"The central claim is that a defocused image can be decomposed into a sharp base set of 3D Gaussians plus M sets of circle-of-confusion Gaussians whose placement is dictated by photographic defocus geometry rather than by free-form kernel learning. For each base Gaussian with mean $\\mu_B$, the CoC diameter is approximated as $\\sigma(\\mu_B) \\approx K |1/d(\\mu_B) - 1/d_F|$, where $d(\\mu_B)$ is the camera-to-Gaussian depth, $d_F$ is a learnable focus-plane distance per image, and $K = f \\times D$ is a learnable aperture scalar. Each CoC Gaussian mean is an offset $\\mu_{\\text{CoC}} = \\mu_B + (\\sigma(\\mu_B)/2)\\,\\beta_m d_m$ with learned direction vectors and scaling factors, and the final defocused image is a softmax-weighted sum of images rendered from the base and CoC Gaussians. At inference the CoC Gaussians are set aside, so sharp novel views, aperture control, and focus-plane control all emerge from a model trained only on defocused images.","pith_inferences":["The depth used in the CoC formula is Euclidean distance from the camera center to the Gaussian mean, which overestimates true optical depth for off-axis points; replacing it with ray-Gaussian intersection depth would be a direct testable extension.","Because $\\beta_m$ is bounded above by 1, the adaptive mechanism can shrink an overestimated CoC but cannot grow an underestimated one, so a two-sided scaling or a depth regularizer is a natural next step.","The learned per-image focus plane suggests a post-hoc focal-stack application: refocused renders could be validated against images actually captured at different focus distances.","The same CoC-Gaussian spawning strategy could be combined with motion-blur kernels to handle scenes with both defocus and camera shake."],"forward_implications":["Rendering with only the base Gaussian set yields sharp images, turning the method into a render-time deblurring tool.","Changing $K$ at inference produces shallower or deeper depth of field, and changing $d_F$ refocuses the image, without retraining.","Because the CoC formula also describes sub-pixel blur, the model works on all-in-focus images and reports higher quality than plain 3DGS on the NeRF-LLFF benchmark.","On the Deblur-NeRF synthetic and real datasets and the DoF-NeRF real dataset, the paper reports higher quantitative scores than the compared ray-tracing and rasterization baselines."],"supporting_citations":[{"why":"Supplies the photographic circle-of-confusion formula that the paper approximates as Eq. (7).","marker":"[7]"},{"why":"Provides the base 3D Gaussian representation and rasterizer that all rendering builds on.","marker":"[9]"},{"why":"First applies a physical CoC model to defocus blur in scene representation and supplies the Eq. (7) approximation and a real benchmark.","marker":"[41]"},{"why":"Defines the blurry-image scene-reconstruction task and provides the synthetic and real defocus datasets used for evaluation.","marker":"[18]"},{"why":"Inspires the flexible scaling and quaternion adjustments for Gaussian deblurring and the densification recipe.","marker":"[10]"},{"why":"Provides the blur-agnostic Gaussian baseline and the weighted-sum strategy that the final rendering uses.","marker":"[26]"},{"why":"COLMAP supplies the camera poses and initial point cloud that initialize the Gaussians.","marker":"[29,30]"}],"fun_headline_variants":["Defocus blur physics yields sharp 3D scenes","Gaussian splatting that sees through defocus","Circle of confusion powers sharp 3D from blur","Turning blurry photos into focus-controlled 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The straight-line distance from the camera center to each Gaussian center is treated as the optical depth that sets the circle of confusion, so scene points away from the optical axis get blur sizes from a proxy that can be systematically wrong, and a single learned aperture scalar cannot correct that off-axis bias.","fun_headline_variants_meta":{"raw":{"variants":["Defocus blur physics yields sharp 3D scenes","Gaussian splatting that sees through defocus","Circle of confusion powers sharp 3D from blur","Turning blurry photos into focus-controlled 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1346,"prompt_tokens":987,"completion_tokens":359,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":296}},"tokens_in":603,"tokens_out":359,"duration_ms":3993,"temperature":1.0,"reasoning_tokens":296,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:52:22.983511+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a synthetic scene containing a planar target tilted about 45 degrees to the optical axis with a known thin-lens aperture, train CoCoGaussian on the resulting defocused views, and compare the recovered CoC diameters of off-axis Gaussians with the ground-truth lens equation: a systematic mismatch that grows with radial distance would confirm that the Euclidean-depth proxy, rather than the learned parameters, is driving the error.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the photographic circle-of-confusion formula that the paper approximates as Eq. (7)."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Provides the base 3D Gaussian representation and rasterizer that all rendering builds on."},{"cited_title":"Dof-nerf: Depth-of-field meets neural radi- ance fields","cited_arxiv_id":null,"evidence_quote":"First applies a physical CoC model to defocus blur in scene representation and supplies the Eq. (7) approximation and a real benchmark."},{"cited_title":"Deblur-nerf: Neural radiance fields from blurry images","cited_arxiv_id":null,"evidence_quote":"Defines the blurry-image scene-reconstruction task and provides the synthetic and real defocus datasets used for evaluation."}],"review_version":1}