{"id":"ed3eb470-6aac-4b4f-9614-7a71a8318e32","arxiv_id":"2412.02225","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"IPSM-Gaussian rectifies the rendered image distribution with warped-view inpaintings, decomposes the SDS objective, and achieves state-of-the-art quality on LLFF and DTU with 3 input views.","lead":"This paper introduces IPSM, a training-free way to use pretrained diffusion models as visual priors for 3D reconstruction from just three input photos, and pairs it with 3D Gaussian Splatting in a pipeline called IPSM-Gaussian. A generalist might care because sparse-view 3D reconstruction could make photorealistic scene capture practical with a handful of images instead of hundreds.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed decomposition of the SDS objective (Eq. 11-13) is not formally valid: the gradient of the second KL omits the dependency of the rectified distribution on θ through the warped image, and the squared-error loss in Eq. 13 is not a decomposition of the SDS loss.","rationale":"The reader's weakest assumption concerned the accuracy of the warped inline prior and its reliance on monocular depth. That is a valid and acknowledged limitation. However, the more load-bearing concern is formal: the central novelty of the paper, the decomposition of the SDS objective, is asserted but not derived. If the gradient in Eq. 12 is not the gradient of the stated two-KL objective, then the method's theoretical contribution is unsupported, leaving the empirical gains (which are real, with ablations showing IPSM helping) to be explained by a heuristic combination of inpainting-conditioned SDS, depth regularization, and geometry consistency. The empirical results and ablations, including the three-run means and standard deviations, are credible enough that the verdict should remain conditional rather than move to accept or reject. The concrete analytical re-derivation and the stop-gradient experiment would settle whether the formal concern actually lands. This is a partial agreement with the reader: they flagged the formal weakness but chose the warped-image accuracy as the weakest assumption; I argue the gradient decomposition is the more fundamental gap, since it directly concerns the central claim rather than an implementation detail or external prior dependency.","tokens_in":25553,"tokens_out":11122,"duration_ms":116754,"concrete_test":"Re-derive the gradient of Eq. 11 with respect to θ, treating the rectified distribution's conditioning on the inline prior as θ-dependent through Eq. 8-10. If the resulting gradient contains additional terms such as ∂I^{i→j}_0/∂D_j · ∂D_j/∂θ, then Eq. 12 is incomplete. Empirically, run the released code with the second sub-objective's gradient blocked from flowing through the warping and depth (i.e., stop-gradient on the inline prior), and compare against the full implementation; if metrics are identical, the second term is already treated as independent of θ, confirming that Eq. 12 does not follow from Eq. 11.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that IPSM 'decomposes the original optimization objective of SDS' (Abstract; Sec. 3.2) by splitting it into two KL divergences and corresponding sub-objectives. This decomposition must be mathematically sound for the claimed contribution to hold. Two distinct problems appear. First, Eq. 12 computes the gradient of the second KL, DKL(˜q_{θ,φ} || p*), as E[ω(t)(ϵ* − ϵϕ) ∂g/∂θ]. But the rectified distribution ˜q_{θ,φ} is conditioned on the inline prior, which is constructed by warping a seen image through the rendered depth D_j (Eq. 8-10). Therefore ˜q depends on θ through D_j, and the gradient of the second KL should contain a term involving ∂I^{i→j}_0/∂D_j · ∂D_j/∂θ, not just the rendering Jacobian ∂g/∂θ. Unless a stop-gradient is applied to the inline prior (which the paper does not state), Eq. 12 is not the gradient of Eq. 11. Second, Eq. 13 defines the IPSM regularization as a sum of squared score differences: ηr||ω(ϵϕ − ϵ)||² + ||ω(ϵ* − ϵϕ)||². For ηr=1, this equals ||ω(ϵ* − ϵ)||² + 2ω²(ϵϕ − ϵ)·(ϵ* − ϵϕ). The cross term is generally non-zero, so the IPSM loss is not a decomposition of the SDS loss; it is a new heuristic objective. The gradient identity in Eq. 12 is an algebraic split of the SDS gradient (since (ϵ* − ϵ) = (ϵϕ − ϵ) + (ϵ* − ϵϕ)), but an identity on the gradient does not imply the corresponding loss decomposes as claimed. Thus the theoretical framing of the central claim is not established; the method may work empirically, but its stated mechanism 'rectifying the distribution' and 'decomposing the SDS objective' does not follow from the equations as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses sparse-view novel view synthesis by combining 3D Gaussian Splatting with a diffusion prior. The authors observe that vanilla Score Distillation Sampling (SDS) degrades reconstruction quality under sparse views, and they attribute this to mode deviation. They propose Inline Prior Guided Score Matching (IPSM), which warps a visible input image to pseudo viewpoints using the current rendered depth, masks the result with a geometric consistency check, and uses this masked warped image as an inpainting condition for a diffusion model. The claimed theoretical contribution is that the SDS objective can be decomposed into two KL divergences, leading to two sub-objectives whose gradients are given in Eq. (12) and whose losses are given in Eq. (13). The full pipeline, IPSM-Gaussian, adds monocular depth regularization and pixel-level geometry consistency regularization. Experiments on LLFF and DTU report state-of-the-art metrics, with ablations and multiple repeated runs. The central theoretical derivation, however, has gaps that bear directly on the paper's main claim of decomposing the SDS objective; my recommendation is major revision.","tokens_in":26069,"tokens_out":7634,"duration_ms":82653,"significance":"If the empirical results hold, the paper offers a practical recipe: using an inpainting diffusion model conditioned on inline geometric priors, together with depth and geometry regularizers, improves sparse-view 3DGS reconstruction without fine-tuning the diffusion model. The paper is commendable for reporting repeated experiments with standard deviations, for including ablations of each loss component, and for releasing code. The additional experiments with 6 and 9 views, with two-view opposite and extrapolation settings, and with view-conditioned diffusion priors, are useful for situating the method. However, the paper's headline claim is that IPSM 'decomposes the original optimization objective of SDS,' and this claim is the basis for the method's motivation and for its distinction from a purely heuristic loss. Because the derivation in Eqs. (11)-(13) is not mathematically established, the contribution currently rests on an empirical heuristic whose theoretical framing needs substantial rework. The empirical improvements are suggestive but do not, by themselves, validate the decomposition claim.","major_comments":[{"comment":"The claimed gradient of the second KL term is incomplete. The rectified distribution q̃^{θ,φ} is conditioned on the warped masked image I^{i→j}_0 and the consistency mask M^{i→j} (Eqs. 8-10), and both depend on the rendered depth D_j, which is a function of θ. Therefore the derivative of D_KL(q̃^{θ,φ} || p*) with respect to θ includes additional terms through the conditioning variable, not just the rendering Jacobian ∂g/∂θ. A stop-gradient on the inline prior is nowhere stated. Without such a device, Eq. (12) is not the gradient of Eq. (11), and the central claim that IPSM 'decomposes the original optimization objective of SDS' is not established. Please derive the missing terms, or explicitly state and justify a stop-gradient or other relaxation that makes Eq. (12) valid.","section":"Sec. 3.2, Eq. (12)"},{"comment":"The squared-score objective is not a decomposition of the SDS loss. Writing a = ω(ε_φ − ε) and b = ω(ε* − ε_φ), one has ||a||² + ||b||² = ||a+b||² − 2a·b, so the cross term is generally non-zero; the objective in Eq. (13) therefore differs from the squared SDS residual by a generally non-vanishing term. In addition, with η_r = 0.1 the two terms are not even on equal footing, so the expression is not an algebraic split of the SDS gradient. Moreover, the gradient of Eq. (13) includes ∂ε_φ/∂θ terms through x_t and through the conditioning image, so Eq. (13) is not equivalent to Eq. (12) under the usual Jacobian-free SDS approximation. Please state precisely what objective is optimized in practice and how it relates to Eq. (11); if the method is a heuristic combination of score-matching terms, that should be stated explicitly rather than presented as a decomposition.","section":"Sec. 3.2, Eq. (13)"},{"comment":"The optimization target is not fixed but is generated from the current θ: the warped image I^{i→j}_0 and the mask M^{i→j} use the rendered depth D_j, which is the very quantity being optimized. This makes the KL decomposition in Eq. (11) circular in the sense that the model is pulled toward a distribution that depends on its own estimate. The monocular depth regularization in Eqs. (14)-(15) mitigates the risk but does not eliminate it, and the paper does not analyze how sensitive IPSM is to inaccurate D_j beyond the illustrative Fig. 6(a). Please provide a quantitative study of this dependence, for example by comparing IPSM with rendered depth versus fixed monocular depth, or by corrupting D_j with controlled errors, to demonstrate that the method is not dominated by self-confirmation.","section":"Sec. 3.2 and Sec. 3.3, Eqs. (8)-(10) vs. Eq. (11)"},{"comment":"The SOTA comparison is only partially controlled. Several baselines are imported from prior papers rather than re-run under the same schedule, initialization, and codebase (e.g., FSGS on LLFF, DNGaussian on DTU), and the FSGS baseline on DTU is reproduced with modified hyperparameters plus an added background-mask prior. These choices are described in Appendix A.4, but the reader cannot tell from Table 1 which numbers are exactly comparable. Please mark clearly which baselines are reproduced under identical conditions, which are taken from published sources, and what effect the added prior has on the reproduced FSGS numbers; ideally, re-run the most competitive baselines under the paper's training schedule and SfM initialization.","section":"Sec. 4.2 and Appendix A.4"}],"minor_comments":[{"comment":"In the second KL term, the distribution p*_0(x^j_t) appears to mix an image-level distribution with a noisy variable x^j_t; this should presumably be p*_t(x^j_t).","section":"Sec. 3.2, Eq. (11)"},{"comment":"The phrase 'without any fine-tuning or pre-training' is stronger than what is implemented, since the method uses a pre-trained Stable Diffusion inpainting model and a pre-trained monocular depth estimator; please clarify that the claim refers to no fine-tuning of the 3D representation or the diffusion backbone.","section":"Abstract and Sec. 1"},{"comment":"The caption of Fig. 6(b) is difficult to parse; please rewrite it to label clearly which panels correspond to Base, IPSM, depth regularization, and the combined method.","section":"Fig. 6(b)"},{"comment":"The phrase 'comprehended analysis' in the Conclusion should be 'comprehensive analysis.'","section":"Sec. 5"},{"comment":"The statement that the FSGS DTU reproduction used MipNeRF-360 hyperparameters because 'we observe that the selected hyper-parameters are more suitable' should be accompanied by the actual values or a small sensitivity study, since this choice directly affects the comparison in Table 1.","section":"Appendix A.4"}],"recommendation":"major_revision","confidential_remarks":"The paper contains a useful empirical recipe and a substantial set of experiments, but the theoretical framing in Sec. 3.2 is load-bearing for the paper's main contribution and is currently not valid as written. I would gate acceptance on a careful revision of the derivation: either supply the missing gradient terms and justify the stop-gradient or approximation, or explicitly reframe IPSM as a heuristic score-matching objective and adjust the claims accordingly. The empirical comparison should also be tightened by marking exactly which baselines are reproduced versus imported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Paper: arXiv:2412.02225. I read it after the stress-test. The empirical core is real: the method works, the ablations are clean, and the comparison to SDS directly shows the failure mode they're fixing. I agree with the reader's take on significance: within sparse-view NVS, beating FSGS and DNGaussian on LLFF and DTU with a no-fine-tuning diffusion prior is a meaningful step. The code release and repeated experiments help.\n\nBut the theory as written does not support the central claim. Eq. 11-13 have two problems. First, the rectified distribution q̃ depends on θ through the warped image (Eq. 8-10), so the gradient of the second KL in Eq. 12 omits the ∂I/∂D_j · ∂D_j/∂θ terms. Unless you stop-gradient the inline prior, and they don't say that, Eq. 12 is not the gradient of Eq. 11. Second, Eq. 13 defines L_IPSM as a sum of two squared score errors, but the gradient of that sum (even under the usual stop-gradient convention) does not give the two terms in Eq. 12. The second term would require differentiating through ε_φ, which they don't. So the 'decomposition of the SDS objective' is an algebraic identity on the gradient, not a loss decomposition. The method is a new heuristic objective, and it's fine to present it as such, but the paper overclaims.\n\nThe circularity concern is real but not fatal: the warped image uses the current depth, so the target can chase its own errors. The monocular depth regularization and the frozen inpainting model put a floor under it. The ablations show the depth term matters, which is consistent with that worry. I'd call it a known limitation rather than a hidden flaw, since they acknowledge it.\n\nThe empirical work holds up: three runs, standard deviations, ablations of each term, extra experiments with 6 and 9 views, and the MipNeRF-360 two-view tests. The SDS instability analysis is a useful addition. The treatment of view-conditioned priors (Zero-1-to-3, ZeroNVS) is honest and shows the method isn't trivially replaced by a stronger prior.\n\nSo: the contribution is the empirical recipe plus the insight that warping yields a useful inpainting condition for score distillation. The derivation needs a proper rewrite—either stop-gradient and justify it, or reframe as a heuristic loss. I'd send this to review, but the reviewers should insist on fixing the theory. The paper deserves a serious referee.","headline":"Strong empirical method built on an invalid gradient decomposition; worth a careful revision, not a desk reject.","tokens_in":26584,"tokens_out":3460,"would_cite":true,"duration_ms":34709,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Score Distillation Sampling can provide effective visual guidance for sparse-view 3D reconstruction if the rendered image distribution is first rectified by warping the sparse views into pseudo viewpoints.","keywords":["score distillation sampling","sparse-view novel view synthesis","diffusion priors","3D Gaussian Splatting","inline priors","mode deviation","inpainting diffusion model","depth regularization"],"falsifier":"Replace the rendered depth in Eqs. 7-10 with increasingly corrupted depth (or remove monocular depth regularization entirely) on a scene with thin structures and repeated texture, and track reconstruction quality: if quality does not degrade, the inline prior is not playing the claimed corrective role; if even mild corruption collapses the gain over SDS, then the external depth prior, not the diffusion rectification, is the load-bearing component.","tokens_in":25389,"feed_emoji":"🎯","tokens_out":6090,"duration_ms":57105,"temperature":0.7,"pith_summary":"The paper claims that Score Distillation Sampling (SDS) fails for sparse-view 3D reconstruction because it drives the rendered image distribution toward the nearest mode of a pre-trained diffusion prior, which is often a failure mode inconsistent with the given views. It proposes Inline Prior Guided Score Matching (IPSM), which first warps a seen view into pseudo viewpoints using the current rendered depth and masks the result for consistency, forming an inline visual prior, then rectifies the rendered distribution with this prior through an inpainting diffusion model. The original SDS objective is decomposed into two sub-objectives: pulling the rendered distribution toward the rectified distribution and pulling the rectified distribution toward the diffusion prior. On top of 3D Gaussian Splatting, the IPSM-Gaussian pipeline adds monocular depth and geometry consistency regularizations and reports state-of-the-art reconstruction quality on LLFF and DTU under three input views. A sympathetic reader would take away that diffusion visual guidance can be harvested under sparse views without fine-tuning or pre-training, provided the geometry used to build the inline prior is trustworthy.","feed_headline":"Rectified render distributions unlock sparse-view diffusion priors","feed_subtitle":"Warped seen views fix SDS's mode deviation, letting diffusion guide 3D reconstruction without fine-tuning or pre-training.","key_machinery":"The rectified distribution, built by inverse warping a seen viewpoint image to pseudo viewpoints with the rendered depth and masked by a depth-consistency threshold (Eqs. 8-10), is the central object. This distribution is the bridge: it decomposes the SDS objective into two KL divergences (Eq. 11) and is realized in practice by an inpainting diffusion model that shares a VAE feature space with the base diffusion model. The two gradient terms of Eq. 12 implement the decomposition: one matches the rendered image to the rectified image, the other matches the rectified image to the diffusion prior.","core_discovery":"The central claim is that the mode deviation of SDS under sparse views can be corrected by a rectified distribution constructed from inline priors: sampling pseudo viewpoints around a seen view, inverse-warping the seen image with rendered depth, masking by depth-consistency, and using the masked warped image as an inpainting condition. IPSM then replaces the single SDS gradient with two matched gradients: one between the rendered noisy image and the rectified noisy image, and one between the rectified noisy image and the pre-trained diffusion prior. The rectified distribution acts as an intermediate state that amplifies the distance between target and failure modes, steering optimization toward the target mode. The authors additionally support the inline prior with monocular depth regularization and a pixel-level geometry consistency loss, and show that direct SDS guidance degrades the 3D Gaussian Splatting baseline performance while IPSM improves it.","pith_inferences":["The need for the rectified distribution to live in the same VAE feature space as the diffusion prior couples the method to a specific inpainting model; a natural extension is to learn a lightweight adapter that maps other inpainting or generation priors into that space.","Because the inline prior is constructed from rendered depth, the method inherits the failure modes of monocular depth estimation; scenes with repeated texture, thin structures, or large occlusions could be tested as stress cases, and one would predict quality drops tied to depth accuracy.","The two-term decomposition suggests a general recipe: any task where a cheap geometric prior can pin down part of the target distribution could use the same rectified-bridge trick to make a text-conditioned generative prior obey scene-specific constraints.","The experiments with opposite and extrapolated views show large absolute gaps remain; the method's benefit is currently largest in near-novel views close to the seen viewpoints, so extending the pseudo-view sampling strategy to cover large baselines is a testable next step."],"forward_implications":["IPSM provides effective diffusion visual guidance for sparse-view reconstruction without fine-tuning or pre-training, improving 3D Gaussian Splatting baselines on LLFF and DTU under 3 views.","Direct SDS with both low and high CFG values degrades or gives unstable reconstruction, whereas IPSM reduces cross-run variance in SSIM, LPIPS, and PSNR.","The method scales to 6 and 9 input views with the same hyperparameters, outperforming prior methods including those that require pre-training.","Depth and geometry consistency regularization are necessary supports: monocular depth keeps the warped inline prior accurate, and masked L1 narrows the pixel-level gap between rendered and rectified images.","View-conditioned 3D diffusion priors used with vanilla SDS still fail to boost sparse-view reconstruction, suggesting that the inline rectification, not the dimensionality of the prior, is what matters."],"supporting_citations":[{"why":"Defines the SDS objective and mode-seeking interpretation that the paper analyzes and decomposes.","marker":"[23]"},{"why":"Provides the 3D Gaussian Splatting backbone and differentiable rendering function used throughout.","marker":"[2]"},{"why":"Supplies the inverse-warping and consistency-mask technique used to build the inline priors.","marker":"[46]"},{"why":"The pre-trained inpainting diffusion model that realizes the rectified distribution in the shared VAE feature space.","marker":"[63]"},{"why":"Provides the monocular depth regularization approach and serves as a strong sparse-view 3DGS baseline to compare against.","marker":"[9]"},{"why":"A representative diffusion-prior method requiring pre-training; IPSM is compared with it and shown to outperform it without pre-training.","marker":"[15]"},{"why":"Establishes the sparse-view dataset protocol on LLFF and DTU and is a baseline for reconstruction quality.","marker":"[7]"},{"why":"Provides the dataset protocol and baseline results for sparse-view NeRF reconstruction.","marker":"[8]"}],"fun_headline_variants":["IPSM: Rectifying SDS to unlock sparse-view diffusion priors","Warped views as inline priors fix sparse-view 3D SDS mode","No fine-tuning: diffusion prior guides sparse-view reconstruction","Sparse-view SDS mode deviation solved by rectified distributions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the inverse-warped, depth-masked image from a seen view is a trustworthy visual condition for the inpainting diffusion model; if the rendered depth is wrong, the inline prior is wrong and the diffusion guidance will reinforce incorrect geometry rather than correct it.","fun_headline_variants_meta":{"raw":{"variants":["IPSM: Rectifying SDS to unlock sparse-view diffusion priors","Warped views as inline priors fix sparse-view 3D SDS mode","No fine-tuning: diffusion prior guides sparse-view reconstruction","Sparse-view SDS mode deviation solved by rectified distributions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1491,"prompt_tokens":939,"completion_tokens":552,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":479}},"tokens_in":555,"tokens_out":552,"duration_ms":5785,"temperature":1.0,"reasoning_tokens":479,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:42:08.327477+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the rendered depth in Eqs. 7-10 with increasingly corrupted depth (or remove monocular depth regularization entirely) on a scene with thin structures and repeated texture, and track reconstruction quality: if quality does not degrade, the inline prior is not playing the claimed corrective role; if even mild corruption collapses the gain over SDS, then the external depth prior, not the diffusion rectification, is the load-bearing component.","supporting_citations":[{"cited_title":"Geconerf: Few-shot neural radiance fields via geometric consistency,","cited_arxiv_id":null,"evidence_quote":"Supplies the inverse-warping and consistency-mask technique used to build the inline priors."},{"cited_title":"High-resolution image synthe- sis with latent diffusion models,","cited_arxiv_id":null,"evidence_quote":"The pre-trained inpainting diffusion model that realizes the rectified distribution in the shared VAE feature space."},{"cited_title":"Dreamfusion: Text-to-3d using 2d diffusion,","cited_arxiv_id":null,"evidence_quote":"Defines the SDS objective and mode-seeking interpretation that the paper analyzes and decomposes."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering,","cited_arxiv_id":null,"evidence_quote":"Provides the 3D Gaussian Splatting backbone and differentiable rendering function used throughout."},{"cited_title":"Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization,","cited_arxiv_id":null,"evidence_quote":"Provides the monocular depth regularization approach and serves as a strong sparse-view 3DGS baseline to compare against."},{"cited_title":"Reconfusion: 3d reconstruction with diffusion priors,","cited_arxiv_id":null,"evidence_quote":"A representative diffusion-prior method requiring pre-training; IPSM is compared with it and shown to outperform it without pre-training."},{"cited_title":"Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs,","cited_arxiv_id":null,"evidence_quote":"Establishes the sparse-view dataset protocol on LLFF and DTU and is a baseline for reconstruction quality."},{"cited_title":"Freenerf: Improving few-shot neural rendering with free frequency regularization,","cited_arxiv_id":null,"evidence_quote":"Provides the dataset protocol and baseline results for sparse-view NeRF reconstruction."}],"review_version":1}