{"id":"4583cc7d-9a20-4238-bfb0-ef108578297b","arxiv_id":"2504.20040","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Augmenting incremental Structure-from-Motion with monocular depth and normal priors makes 3D reconstruction robust in low-overlap, low-parallax, and high-symmetry scenes.","lead":"This paper builds a Structure-from-Motion system that uses monocular depth and surface-normal estimates from neural networks to reconstruct 3D scenes from very few, low-overlap images. It reports large accuracy gains over existing pipelines in extreme viewpoint and symmetry-heavy settings, with code made public.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The global-scale correction in Eq. (1) is the load-bearing link in the two-view-only pipeline, but the paper never tests robustness to depth bias that is not a pure per-image scale.","rationale":"The Reader's CONDITIONAL verdict is well aligned with what I find. The central claim -- accurate two-view-only reconstruction from monocular priors -- is plausible and strongly supported by the triplet experiments and across four datasets with multiple matchers and depth models. The code is public, and the ablation study shows that lifting and depth refinement are necessary components, so the core mechanism is not an artifact. My main concern is narrower than the Reader's general worry about uncertainty calibration: the specific correction in Eq. (1) only compensates a single global scale per image, while Cint in Eq. (5) then trusts the scaled prior. Real monocular depth models exhibit offset and depth-dependent bias, not only scale error. The paper does not test this failure mode, and its own Appendix C calibration plots show that the confidence estimates are imperfect even after rescaling. The GT-depth ablation in Table 5 is the closest evidence: GT LiDAR depth (unbiased but sparse) raises AUC@1° substantially, but leaves AUC@20° roughly flat, indicating that the refinement cannot completely remove bias in the prior. This is a load-bearing concern because the empirical payoff of the method in the advertised low-overlap regime comes precisely from trusting lifted single-view points; if the first anchor depth is biased in a non-scale way, those points provide systematically wrong geometry. The proposed test is cheap, uses the released public code, and directly measures sensitivity to non-scale bias. If the sensitivity is small, the central claim stands and only the reporting of threshold values and error bars remains. If sensitivity is large, the paper should qualify its robustness claims to priors whose bias is approximately a global scale. Either way the verdict remains CONDITIONAL rather than a rejection, since the claimed experiments are internally consistent and the open issue is an untested assumption rather than a demonstrated contradiction.","tokens_in":22362,"tokens_out":13699,"duration_ms":165863,"concrete_test":"Take an ETH3D minimal-overlap scene and rerun the released MP-SfM pipeline with Metric3D-v2 priors deliberately contaminated by an additive offset: D' = D + b * median(D), with b in {-0.10, -0.05, -0.02, 0, +0.02, +0.05, +0.10}, re-estimating the Eq. (1) scale factor each time. If AUC@1°/5°/20° moves by more than a few points for |b| >= 2%, the method depends on the pure-global-scale assumption and the robustness claim for generic monocular depth priors is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that monocular depth priors can replace three-view tracks. The mechanism assumes each prior depth map is correct up to a single per-image scale: Eq. (1) divides by a median ratio and the whole refinement then treats the scaled prior as a strong uncertainty-weighted constraint via Cint in Eq. (5). If an off-the-shelf model produces bias that is not a pure scale -- an additive offset, a depth-dependent scale drift, or spatially varying error -- the median in Eq. (1) cannot remove it, and the network-predicted uncertainties do not model it. The paper's own Appendix C states that Metric3D-v2 uncertainties are overconfident for the most confident pixels and must be rescaled by a tuned constant, clipped at 2 cm, and supplemented with a depth-proportional term. The paper's Limitations section similarly concedes that reliable uncertainties are a dependency. Since the headline result is robustness to imperfect off-the-shelf priors in low-overlap scenes, the unstated assumption that priors are globally scale-correct is load-bearing. Table 5's ground-truth-depth ablation is consistent with this concern: replacing the prior with sparse GT depth sharply improves AUC@1° but not AUC@20°, suggesting the refinement stage does not fully repair prior bias.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MP-SfM, an incremental Structure-from-Motion pipeline that augments COLMAP with monocular depth and surface-normal priors, including their predicted uncertainties. The key idea is to lift the classical requirement for three-view tracks: depth-lifted 2D-3D correspondences allow registration of new views with only two-view overlap, while an alternating optimization of bundle adjustment, depth refinement, and normal integration fuses the priors into the reconstruction. A dense depth-consistency check rejects incorrectly registered views, particularly in symmetric scenes. The method is evaluated on ETH3D, SMERF, Tanks and Temples, and RealEstate10k under varying overlap and parallax conditions, showing consistent improvements over COLMAP, GLOMAP, SLR, DF-SfM, VGGSfM, StudioSfM, and MASt3R-SfM. The paper claims that this is the first approach capable of reliably reconstructing challenging indoor environments from few images, while requiring little tuning.","tokens_in":22658,"tokens_out":5468,"duration_ms":54027,"significance":"If the results hold, this is a meaningful advance for incremental SfM: it directly attacks the three-view-track requirement, a known practical bottleneck for non-expert capture, and demonstrates large gains on low-overlap and low-parallax benchmarks. The evaluation is commendably broad: external benchmarks, multiple sparse and dense matchers, several monocular depth models, and component-wise ablations. The public code release is a concrete strength that supports reproducibility. The main robustness claim is credible, but the evidence does not yet cover all the error modes that the paper claims to handle: the per-image scale correction assumes scale-only prior errors, and the calibrated uncertainty machinery relies on several unreported tuning constants. These gaps are fixable but should be addressed before publication.","major_comments":[{"comment":"The median-ratio scaling in Eq. (1) corrects the monocular depth prior only up to a single per-image scale factor. The paper's central claim of robustness to errors in the priors (Abstract; Sec. 1) therefore depends on the unstated assumption that prior depth errors are predominantly scale-only. This assumption is not tested: no experiment perturbs the priors with additive offsets, depth-dependent scale drift, or spatially varying bias. The ground-truth-depth ablation in Table 5 (ETH3D minimal overlap: AUC@1° improves from 27.3 to 42.9) shows that prior bias, not just its scale, limits fine-grained accuracy. Please add a synthetic-bias ablation or explicitly scope the robustness claim to scale-correct priors.","section":"Eq. (1) and Sec. 3.1"},{"comment":"The depth consistency check is a central safeguard against symmetry failures, yet the two decision parameters—gamma in Eq. (6) and the ratio beta_hat mentioned in the text—are never given numeric values. Since Table 6 shows this check is crucial in the SMERF scenes, leaving these thresholds unreported prevents reproduction and makes it impossible to judge how much tuning the method requires.","section":"Sec. 3.4, Eq. (6)"},{"comment":"The uncertainty calibration and robust-loss configuration involve several tuned quantities—the constant scaling factor for predicted uncertainties, the 2 cm standard-deviation clip, the depth-proportional uncertainty factor, and the robust loss scales for Creg and Cint—but the final values are not reported. Because the claim of 'principled uncertainty propagation' and little tuning (Abstract; Sec. 1) is part of the contribution, the paper should list all free parameters and the data splits used to select them.","section":"Appendix C and Sec. 4.3"}],"minor_comments":[{"comment":"The sentence 'We consider a view c as inconsistent if any of the overlapping views' beta_i exceeds a ratio beta_hat of occluded pixels' is ambiguous: Eq. (6) defines beta_i as a ratio of inconsistent pixels, so the phrase 'ratio of occluded pixels' should be clarified or removed.","section":"Sec. 3.4"},{"comment":"In the definition of Sigma_r, the last diagonal entry is written as sigma^2_{N-_u}; it should presumably be sigma^2_{N-_v}.","section":"Eq. (7) / Appendix B"},{"comment":"There are several typos in the caption: 'estiamtes', 'yileded', and 'uncertianties' should be corrected.","section":"Fig. 7 caption"},{"comment":"The header of the right block is garbled in the manuscript ('max overlapminimal, 0% <5% <10% <30%'); please fix the column labels to make the overlap buckets unambiguous.","section":"Table 1"},{"comment":"The sentence 'The GT camera poses were estimated with COLMAP – achieving sufficient accuracy by using 10 to 100 times more images' is missing a subject; it should read 'The GT camera poses were estimated with COLMAP using 10 to 100 times more images, which we assume achieves sufficient accuracy.'","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MP-SfM is worth a serious look. It does something concrete: replaces the three-view track requirement in incremental SfM with monocular depth and normal priors, and the results on ETH3D, Tanks & Temples, SMERF, and RealEstate10k back that up. The two-view-only registration, dense depth consistency check for symmetry rejection, and uncertainty-weighted normal integration are genuine additions over StudioSfM, COMO, and MASt3R-SfM. The code is public, which makes the empirical claims checkable.\n\nThe soft spots are modest but real. The largest is the one the stress-test flags: Eq. (1) assumes each depth prior is correct up to a single per-image scale. That is a strong assumption. A depth model with additive offset or depth-dependent drift will not be fixed by a median ratio, and the network uncertainties won't model it either. The paper's own Appendix C admits Metric3D-v2 is overconfident on the most confident pixels and needs clipping plus a depth-proportional term, which is a patch rather than a principled model of bias. The GT-depth ablation in Table 5 is consistent with this: sparse unbiased depth helps a lot at AUC@1° but less at AUC@20°. The stress-test slightly overstates the AUC@20° reading—it does improve—but the underlying point stands: the paper never directly tests robustness to non-scale bias. A synthetic experiment adding known bias to priors would settle this. That is a revision request, not a fatal flaw, because the robust losses, uncertainty clipping, and experiments with Depth Pro and Depth Anything V2 show the pipeline tolerates a fair amount of error.\n\nTwo smaller issues. The depth consistency thresholds (gamma, beta_hat) and the calibration factors are not reported in the main text; the paper says they are tuned but doesn't give values. And there are no error bars on the AUC numbers across the five sampled collections per scene, so we can't tell if the differences between ours and the next best are meaningful. These are easy to fix.\n\nThe 'first capable of reliably reconstructing challenging indoor environments' claim is slightly strong, but the SMERF results justify most of it against MASt3R-SfM. The limitations section is unusually candid about the uncertainty dependency, which I credit.\n\nVerdict: send it to peer review. It is a substantial systems contribution with reproducible evidence and a clear central claim. The referee should push for the bias experiment and the missing numbers, but the architecture and results are solid.","headline":"MP-SfM is a real advance in incremental SfM for low-overlap scenes; the main soft spot—untested robustness to non-scale depth bias—is worth a revision experiment but not a rejection.","tokens_in":23207,"tokens_out":2957,"would_cite":true,"duration_ms":27619,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Monocular depth and normal priors let incremental SfM reconstruct scenes from two-view tracks alone.","keywords":["structure-from-motion","monocular depth priors","surface normals","uncertainty propagation","two-view reconstruction","incremental SfM","low-overlap reconstruction","depth consistency check"],"falsifier":"Take a scene with strictly zero three-view overlap and run the full pipeline with the monocular depth term disabled (the paper's 'no lifting' ablation); the reported minimal-overlap AUC at 5 degrees falls from about 56 to 16 on the indoor benchmark, while the pose error with priors enabled is close to ground truth. A reader can reproduce that contrast on a held-out set of such triplets: if accurate poses persist without the depth prior, the three-view requirement was never the bottleneck; if they collapse, the two-view claim rests on the priors, exactly as stated.","tokens_in":22191,"feed_emoji":"🧭","tokens_out":8921,"duration_ms":86445,"temperature":0.7,"pith_summary":"Classical incremental structure-from-motion needs three-view tracks: a new camera can only be registered through 3D points already triangulated by two earlier views, so low-overlap or low-parallax captures routinely break. This paper claims that requirement can be dropped by feeding monocular depth and surface-normal predictions, with their uncertainties, into every stage of the pipeline. The predictions supply per-image 3D scaffolds, letting the system register a new view and grow structure from two-view correspondences alone, while a joint optimization refines poses, sparse points, and dense depth maps together. A dense depth-consistency step rejects misregistrations that sparse geometry accepts, notably those caused by repeated or symmetric structure. The reported result is accurate reconstruction in extreme low-overlap and low-parallax conditions, with competitive accuracy where overlap is plentiful; if true, it substantially widens the class of image sets ordinary users can reconstruct.","feed_headline":"Three-view overlap no longer required for SfM","feed_subtitle":"Fusing monocular depth and normal priors with uncertainty weighting keeps SfM accurate in low-overlap, symmetric scenes.","key_machinery":"The load-bearing mechanism is an uncertainty-weighted fusion of single-view and multi-view geometric constraints, solved by alternating optimization. The objective $C_{BA} + C_{reg} + C_{int}$ couples (i) standard bundle adjustment over sparse 3D points, (ii) a depth-regularization term $C_{reg}$ pulling scene points toward refined per-image depth maps $D^*_i$, and (iii) a bilateral normal-integration term $C_{int}$ that conditions $D^*_i$ on the monocular depth and normal priors with Mahalanobis weighting by predicted covariances. Because the Hessian of the joint cost loses the block-diagonal structure needed for Schur-complement elimination, the authors alternate: refine each depth map independently with $C_{reg}+C_{int}$ at fixed poses and points, then optimize poses and points with $C_{BA}+C_{reg}$ at fixed depth maps. A final dense depth-consistency check, comparing each image's refined depth against a reprojected min-depth buffer of overlapping views, rejects images that contradict free space.","core_discovery":"The paper's central claim is that the scale information incremental SfM normally obtains from three-view tracks can instead be supplied by per-image monocular priors, up to one unknown scale per image. Each view's predicted depth and surface normals act as soft constraints: 3D points lifted from a single view serve as 2D–3D correspondences for pose estimation, and a combined objective $C_{BA}+C_{reg}+C_{int}$ couples sparse bundle adjustment with depth regularization and bilateral normal integration. The predicted uncertainties are propagated and calibrated, so bad depth estimates are down-weighted rather than trusted. A dense forward–backward depth-consistency check then de-registers any image whose refined depth contradicts overlapping views, removing symmetry-induced false positives. On low-overlap subsets of standard benchmarks, the authors report accurate pose estimates for scenes with zero three-view overlap where existing incremental, structure-less, and learned two-view pipelines fail, and they maintain competitive accuracy in dense high-overlap settings; they further state that this makes the approach the first to reliably reconstruct challenging indoor scenes from few images.","pith_inferences":["Beyond the paper, the same uncertainty-weighted fusion should apply to camera relocalization and dense SLAM, where pure rotation, low texture, or repetitive structure make epipolar constraints degenerate; a monocular depth prior could supply scale and depth hypotheses there too.","The per-image scale formulation implies that a monocular model with reliable relative depth but no metric scale could be substituted if its scale is recovered from the first verified two-view pair; that would decouple the method from metric depth models.","The depth-consistency check could be inverted into an active-capture signal: images that repeatedly fail it flag symmetric or ambiguous regions, telling a non-expert user exactly which additional views would disambiguate the scene."],"forward_implications":["A reconstruction can be built from image pairs with no triple overlap, so sparse casual captures no longer need careful planning to guarantee three-view coverage.","Dense two-view correspondences in texture-poor regions become usable directly, improving completeness where sparse keypoints are scarce.","Because the priors are treated as uncertain soft constraints, swapping in a different monocular depth or normal estimator requires little retuning, so future improvements in single-image geometry transfer to SfM.","Symmetry-induced wrong registrations can be detected and removed by dense depth consistency, even when sparse geometric verification accepts them.","In low-parallax configurations, incremental reconstruction approaches the accuracy of global methods, which previously did not suffer from the same failure."],"supporting_citations":[{"why":"The classical incremental SfM pipeline the method extends; supplies the baseline, view-graph handling, bundle adjustment scheduling, and filtering.","marker":"[50]"},{"why":"Supplies the monocular depth and surface-normal priors with uncertainty estimates used in the main configuration.","marker":"[28]"},{"why":"Provides the bilateral normal-integration formulation that couples refined depth maps to the normal priors.","marker":"[11]"},{"why":"Supplies dense two-view correspondences and per-image depth used in the dense-matching configuration.","marker":"[34]"},{"why":"Provides dense two-view matching that exploits texture-poor regions and forms two-view-only tracks.","marker":"[19]"},{"why":"The learned two-view-based SfM baseline that the method is compared against on low-overlap scenes.","marker":"[16]"},{"why":"The learned symmetry-disambiguation approach that the dense depth-consistency check is compared against.","marker":"[9]"}],"fun_headline_variants":["SfM works without three-view overlap","Monocular priors replace triple-view constraints","Depth and normal priors fix low-overlap SfM","No three-view overlap needed for SfM","Uncertainty-calibrated monocular priors robustify SfM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's load-bearing premise is that monocular depth and normal predictions are accurate enough up to a per-image scale that their uncertainty-weighted fusion improves multi-view geometry; if the predicted uncertainties are systematically overconfident or miscalibrated, the joint optimization can pull poses toward wrong depth rather than toward consistent multi-view structure.","fun_headline_variants_meta":{"raw":{"variants":["SfM works without three-view overlap","Monocular priors replace triple-view constraints","Depth and normal priors fix low-overlap SfM","No three-view overlap needed for SfM","Uncertainty-calibrated monocular priors robustify SfM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000433,"raw_usage":{"total_tokens":2224,"prompt_tokens":981,"completion_tokens":1243,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":1165}},"tokens_in":597,"tokens_out":1243,"duration_ms":8189,"temperature":1.0,"reasoning_tokens":1165,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:36:25.313829+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a scene with strictly zero three-view overlap and run the full pipeline with the monocular depth term disabled (the paper's 'no lifting' ablation); the reported minimal-overlap AUC at 5 degrees falls from about 56 to 16 on the indoor benchmark, while the pose error with priors enabled is close to ground truth. A reader can reproduce that contrast on a held-out set of such triplets: if accurate poses persist without the depth prior, the three-view requirement was never the bottleneck; if they collapse, the two-view claim rests on the priors, exactly as stated.","supporting_citations":[{"cited_title":"Structure-from-Motion Revisited","cited_arxiv_id":null,"evidence_quote":"The classical incremental SfM pipeline the method extends; supplies the baseline, view-graph handling, bundle adjustment scheduling, and filtering."},{"cited_title":"Metric3D v2: A Versatile Monocular Geo- metric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the monocular depth and surface-normal priors with uncertainty estimates used in the main configuration."},{"cited_title":"Ground- ing Image Matching in 3D with MASt3R","cited_arxiv_id":null,"evidence_quote":"Supplies dense two-view correspondences and per-image depth used in the dense-matching configuration."},{"cited_title":"RoMa: Robust Dense Feature Match- ing","cited_arxiv_id":null,"evidence_quote":"Provides dense two-view matching that exploits texture-poor regions and forms two-view-only tracks."},{"cited_title":"Doppelgangers: Learning to Disambiguate Images of Similar Structures","cited_arxiv_id":null,"evidence_quote":"The learned symmetry-disambiguation approach that the dense depth-consistency check is compared against."}],"review_version":1}