{"id":"59448f45-1a23-4916-bf3f-42c2aa8d9260","arxiv_id":"2501.04074","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"NeRF-MD automatically detects mirrors from the photometric inconsistencies left by a standard NeRF and reconstructs scenes with explicit mirror primitives, without user-provided masks.","lead":"This paper introduces NeRF-MD, a method that finds mirrors in multi-view photos automatically instead of relying on hand-drawn mirror masks. It trains a standard NeRF first, uses image quality and depth confidence scores to detect mirror regions, then jointly refines the scene and mirror surfaces.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mirror detection relies on an empirical SSIM/depth-confidence coincidence that already fails on a dataset scene; the paper neither measures detection accuracy nor includes that failure in its headline quantitative results.","rationale":"The reader's weakest assumption and my stress-test converge on the same point: Eq. (7) assumes a specific co-occurrence of photometric inconsistency and depth confidence in mirror regions, and the paper's own discussion and Fig. 6 show this co-occurrence is not guaranteed. This is the load-bearing link between the initial standard NeRF and the claimed automatic detection. The paper is honest about the failure mode and even shows a graceful degradation, but it does not quantify detection reliability, and its headline quantitative table is computed on a favorable subset. The central contribution (automatic detection without annotations) therefore holds conditionally on an unmeasured empirical property. The reader already assigned CONDITIONAL with moderate confidence, and I agree; my stress-test does not move the verdict, but it sharpens the condition: the paper should either report detection metrics on the full dataset or explicitly scope the claim to scenes where the initial depth estimate is confident. No internal inconsistency was found in the rendering or optimization equations; the concern is about the validity of the detection signal, not about numerical errors.","tokens_in":18759,"tokens_out":2905,"duration_ms":30727,"concrete_test":"Run the full pipeline on every scene of the TraM-NeRF dataset (including the Fig. 6 scene and the front-facing/non-zero-roughness scenes removed from Table 1), and report per-scene detection accuracy: IoU between the antialiased mirror mask rendered from the fitted primitive and the ground-truth annotation, plus the distance between the fitted plane/cylinder and the true mirror geometry. Also tabulate how many scenes satisfy the Eq. (7) assumption (high inconsistency, low depth variance) by thresholding s(r) against the ground-truth mask. If detection IoU is low or the assumption fails on more than one scene, the central claim should be narrowed to scenes with confident mirror depth estimates.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of automatic mirror detection rests on the score in Eq. (7), s(r) = (1 - SSIM(r))/2 * e^{-c V(r)}, which isolates mirror pixels only if the initial standard NeRF produces both high structural dissimilarity and low depth variance there. This is an empirical property, not a guaranteed consequence of volumetric rendering under mirror reflections. Section 6 admits two failure modes: (i) a mirror can be reconstructed as proxy geometry behind an uncaptured wall, producing no photometric inconsistency; and (ii) the first stage can yield implausible mirror depths, as in Fig. 6, causing the primitive initialization to land far from the actual mirror and the final result to retain artifacts. The quantitative evaluation in Table 1 excludes scenes that are 'front-facing or containing specular surfaces with non-zero roughness,' and the paper never reports mirror detection metrics (e.g., mask IoU or primitive parameter error) on any scene, including the Fig. 6 failure case. Because the joint optimization can shrink or discard erroneous mirrors (Section 6), failures degrade gracefully to baseline quality, so the reported improvements come only from scenes where the heuristic already works. Consequently, the paper demonstrates a pipeline that works when the coincidence holds, but does not establish that mirror detection is reliable across the dataset family it targets.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NeRF-MD, a two-stage pipeline for reconstructing neural radiance fields of scenes with mirroring surfaces without user-provided mirror masks. In the first stage, a standard NeRF is trained with a depth reprojection loss; mirror candidate pixels are then identified using a per-pixel score s(r) = (1 - SSIM(r))/2 * exp(-c V(r)) that combines structural dissimilarity with depth variance. Candidate pixels are unprojected, clustered, and fit with RANSAC primitives. In the second stage, the radiance field and primitive parameters are jointly optimized using reflected rays and differentiable antialiased mirror masks, with a scheduled p-norm photometric loss. The method is evaluated on seven synthetic and two real scenes from the TraM-NeRF dataset, excluding front-facing scenes and surfaces with non-zero roughness, and is compared against NeRF baselines and prior-annotated mirror-aware methods. The paper also documents failure modes, including mirrors reconstructed as proxy geometry and implausible initial depths.","tokens_in":18970,"tokens_out":3969,"duration_ms":41653,"significance":"If the central claim holds, the paper would remove a major practical barrier in mirror-aware neural scene reconstruction, namely the need for manual mirror masks. The work has several genuine strengths: the evaluation is end-to-end on held-out test views across nine scenes, standard deviations are reported, comparisons include methods with strong priors, and the source code is publicly available. The SSIM-plus-depth-variance score is a simple, falsifiable heuristic that is interesting in its own right. However, the central claim of faithful automatic mirror detection is not directly tested: no detection metric (mask IoU, precision/recall, primitive parameter error) is reported, and the paper's own discussion in Section 6 shows that the detection mechanism can fail in ways that are not reflected in the headline aggregate numbers.","major_comments":[{"comment":"The paper's central claim is automatic mirror detection, yet no detection metric is reported anywhere. Table 1 reports only downstream rendering metrics (PSNR, SSIM, LPIPS) on full images and on preset mirror-region masks; there is no IoU, precision/recall, or primitive-parameter error for the detected mirror planes on any scene. Because Section 6 states that erroneous mirrors are quickly shrunk or moved behind scene geometry so that the result falls back to baseline quality, the reported rendering improvements do not by themselves establish that mirrors are faithfully detected. Please report per-scene detection metrics against the TraM-NeRF masks, including the Fig. 6 failure case, and state the metric values for scenes where detection fails.","section":"Section 5, Table 1"},{"comment":"The score s(r) in Eq. (7) isolates mirror pixels only if the initial standard NeRF produces both high structural dissimilarity and low depth variance in mirror regions. This is an empirical coincidence rather than a guaranteed property, and the paper itself documents two failure modes in Section 6: a mirror can be reconstructed as proxy geometry when the region behind the wall is not captured, producing no photometric inconsistency, and Fig. 6 shows a scene where the initial depth is implausible, causing the primitive initialization to miss the mirror and the final result to retain artifacts. The manuscript should quantify how often these failure modes occur across the dataset family and state the conditions under which Eq. (7) is reliable; otherwise the 'faithful detection' claim in the abstract is not supported.","section":"Section 4.1, Eq. (7), and Section 6"},{"comment":"The shape-fitting procedure requires the type and number of primitive shapes k to be provided a priori. This is a scene-level prior that is not inferred from the data. While the method avoids per-pixel annotations, the claim of 'automated localization and reconstruction' in the introduction is weakened by the need to know the number of mirrors and the shape class before fitting. Please either automate the selection of k and shape type or explicitly qualify that the method detects mirror locations only given the mirror count and shape class as inputs.","section":"Section 4.2 and Section 6"},{"comment":"The evaluation excludes scenes that are 'front-facing or containing specular surfaces with non-zero roughness.' This selection narrows the scope of the claims but is not justified or analyzed. Since the stated goal is general scenes containing mirroring surfaces, the paper should either include these excluded cases or provide an analysis of why they are outside the method's intended operating range. As presented, the quantitative conclusions apply only to the selected subset, and the reader cannot assess how the method behaves on the broader dataset.","section":"Section 5"}],"minor_comments":[{"comment":"The p-norm scheduling parameters (tau_init, tau_inc, tau_std), the threshold S, and the exponent c in Eq. (7) are not given in the main text, and no sensitivity analysis is provided; please report the values used and ideally include an ablation.","section":"Section 4.3, Eq. (8), and Fig. 3"},{"comment":"The sentence 'MS-NeRF struggle to handle the scenario of high-order bounces' contains a subject-verb agreement error ('struggle' should be 'struggles'), and the following explanation about network capacity is difficult to parse; please rewrite for clarity.","section":"Section 5.1"},{"comment":"The text uses 'apriori knowledge' where the standard form is 'a priori'; the same typo appears in the description of the primitive fitting procedure.","section":"Section 6"},{"comment":"Reference [33] is formatted as 'Hollandet al.' with a missing space; this should be corrected.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of a computer vision venue and the end-to-end results are promising. The main gap is that the paper's central contribution, mirror detection, is only evaluated indirectly through rendering quality. I do not see a circularity problem, since mirror parameters are fit on training images and evaluated on held-out test views, but the lack of direct detection metrics is a substantive issue that should be addressed before publication. The authors should also make the supplementary material with hyperparameter values available in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"NeRF-MD is worth a look if you work on specular surfaces in neural rendering. The genuinely new thing is the automatic step: instead of requiring mirror masks, it uses the fact that a vanilla NeRF renders mirror regions with low SSIM while depth is confident, unprojects those pixels, fits planar (or primitive) shapes, and jointly optimizes the radiance field and the mirror geometry. That removes an annotation bottleneck that all the prior mirror-aware NeRF/Gaussian methods share. The quantitative story is honestly presented: on the nine-scene subset they beat the no-prior baselines on PSNR/SSIM/LPIPS and land close to TraM-NeRF, which uses ground-truth masks. The mirror-region metrics in particular show a large jump over baselines. The paper also lists its own limitations in Section 6, including a failure case in Fig. 6 where depth initialization is wrong and results degrade.\n\nWith that said, the central claim about detection is not directly measured. There is no IoU, precision-recall, or primitive parameter error anywhere, so \"faithful detection\" is inferred from reconstruction quality, not shown. The stress-test note is on target: the score in Eq. (7) is an empirical heuristic, not a guaranteed property of volumetric rendering, and Section 6 admits the proxy-geometry failure mode where no inconsistency appears. The evaluation also excludes front-facing and non-zero-roughness specular scenes, and the Fig. 6 failure is not in Table 1. The user has to specify the number and type of primitives, which is a meaningful prior even if not a mask. These caveats mean the paper demonstrates a working pipeline when the heuristic holds, but does not establish reliable detection across the target scene family.\n\nReproducibility is a minor concern: a code link is given, but no commit hash, and the lack of detection metrics makes the key claim harder to verify independently.\n\nWho this is for: people building annotation-free pipelines for mirrors and reflective surfaces, and anyone comparing mirror-aware NeRF variants. I would bring it to a reading group and would cite it as the current state of the art for unsupervised mirror localization in NeRFs, with the detection caveat noted.\n\nSend it to peer review. It needs major revision — add detection metrics, include failure cases, and at least ablate the user-supplied primitive count and threshold — but the core idea is new and the end-to-end benefits are real.","headline":"Automatic mirror detection from NeRF artifacts is a real contribution, but the paper sells the detection claim a bit harder than the evidence supports; the end-to-end results are good enough that it deserves a serious referee.","tokens_in":19562,"tokens_out":2266,"would_cite":true,"duration_ms":23181,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A standard NeRF's own failures reveal where mirrors are, and that cue is enough to reconstruct mirrored scenes without user-provided masks.","keywords":["neural radiance fields","mirror detection","specular reflection","structural similarity","surface primitives","joint optimization","novel view synthesis","depth consistency"],"falsifier":"Capture a multi-view scene with a mirror whose reflected content lies entirely outside the training views (for example, a mirror on a wall facing a closed corridor), run the initial NeRF with the depth reprojection loss, and check whether any mirror pixels receive a high score $s(r)$. The paper's own proxy-geometry discussion predicts they will not, which would show the automatic detection is not general across capture configurations.","tokens_in":18519,"feed_emoji":"🪞","tokens_out":5037,"duration_ms":48901,"temperature":0.7,"pith_summary":"Neural radiance fields (NeRFs) usually break on mirrors because a mirror shows each camera a different virtual scene, so no single 3D volume can reproduce it consistently. NeRF-MD turns that failure into a detector: after a first standard-NeRF training run, pixels whose rendered color is structurally dissimilar from the photograph but whose depth estimate is confident are treated as mirror candidates. Those pixels are unprojected to a point cloud, clustered, and fit with planar 3D primitives that explicitly reflect rays in a second training stage. The paper claims this removes the need for mirror masks or other annotations while reaching image quality close to methods that are given exact mirror masks. If correct, mirror handling becomes a byproduct of ordinary NeRF training rather than a separate supervised step.","feed_headline":"Mirrors are found by where NeRFs fail","feed_subtitle":"Pixel scores from SSIM and depth certainty locate mirrors, so reconstruction needs no user-drawn masks.","key_machinery":"The load-bearing object is the per-ray score $s(r)$, built from SSIM against ground truth and the variance of the expected absorption depth. Equally central are: the depth reprojection loss from sparse-view NeRF training, which makes initial geometry plausible enough for unprojection; the clustering and RANSAC-based fitting of candidate pixels into planar primitives; a differentiable antialiased mask that blends primary and reflected ray contributions; and a $p$-norm schedule ($p$ starts at 2, drops to 1, returns to 2) that keeps the optimization from falling into a no-mirror local optimum. The reflected-ray formulation follows the transmittance-aware mirror rendering of TraM-NeRF [33], extended here to be differentiable with respect to mirror parameters.","core_discovery":"The central discovery is that mirror surfaces can be located from the artifacts a standard NeRF produces, provided the initial geometry is constrained by a depth reprojection loss. In mirror regions, the radiance field cannot reconcile the reflected appearances seen from different views; it renders blurry, structurally anomalous images while still placing the alleged surface at a fairly stable depth. The score $s(r) = \\frac{1-\\mathrm{SSIM}(r)}{2} e^{-c V(r)}$ combines local structural dissimilarity with depth variance so high-score pixels mark exactly the inconsistent-but-confident regions, which are fit as explicit mirror primitives. The second stage jointly trains the radiance field and the primitive parameters by tracing reflected rays and blending primary and reflected colors through differentiable antialiased masks, with a norm schedule that moves from $L^2$ to $L^1$ and back to sharpen mirror edges without introducing flat-color artifacts. On the tested synthetic and real-world multi-mirror scenes, the method improves over annotation-free baselines and approaches the quality of supervised mirror-aware methods.","pith_inferences":["Beyond the paper's experiments, the same inconsistency signal could be applied to 3D Gaussian splatting or other radiance-field variants, since the failure mode—mirror reflections being multi-view inconsistent—is representation-agnostic.","If detection relies on the failure signature, scenes with mirrors reflecting unobserved space remain a natural blind spot: the paper itself notes such mirrors can be absorbed as proxy geometry, so a practical deployment would need a complementary cue such as motion parallax or geometric plausibility.","The norm schedule suggests a generic training trick for explicit-geometry NeRF extensions: starting with $L^2$, quickly switching to a sparsity-favoring $L^1$ plateau, and returning to $L^2$ can help jump between local optima, and testing that schedule on other surface types would isolate its role.","The method's upper bound is set by the quality of the initial depth, so using stronger monocular depth priors may make detection work in sparser captures where the current reprojection loss lacks signal."],"forward_implications":["Mirror masks, the main user burden of previous mirror-aware NeRF methods, are no longer required for scenes whose mirrors produce detectable inconsistency.","The detected primitives give an explicit, optimizable mirror geometry that can be refined during training, so the reconstruction of the mirror plane and the scene behind it improve together.","Because the approach routes reflected ray contributions back to actual scene points, it reduces the blurriness and multi-view ghosting that standard NeRFs show in mirror regions, and it improves perceptual metrics on full test images.","The optimization falls back gracefully to a standard NeRF quality level when shape initialization is poor, so a failed mirror estimate degrades performance but does not destroy the scene model.","The method is currently limited to mirrors visible to primary rays and mostly evaluated on planar mirrors, with one cylinder experiment suggesting extension to curved primitives."],"supporting_citations":[{"why":"Supplies the reflected-ray rendering formulation, the transmittance-aware mirror model, and the synthetic and real-world datasets used for evaluation; the paper extends it to be differentiable with respect to mirror parameters.","marker":"[33]"},{"why":"Defines the standard NeRF whose initial training produces the photometric inconsistency signal and whose quality serves as a baseline.","marker":"[63]"},{"why":"Provides the depth reprojection loss used in the initial stage to make geometry plausible enough for mirror detection.","marker":"[92]"},{"why":"Defines the SSIM metric that forms the structural dissimilarity component of the per-ray mirror score.","marker":"[101]"},{"why":"Supplies the differentiable antialiased mask rendering that lets gradients flow to mirror parameters without unstable NeRF gradients.","marker":"[41]"},{"why":"Provides RANSAC, used to fit each segmented cluster to a primitive surface in the detection stage.","marker":"[25]"},{"why":"Provides k-means clustering, used to segment the candidate point cloud into one region per mirror primitive.","marker":"[52]"},{"why":"Serves as the main annotation-free multi-space baseline that the paper compares against and outperforms in mirror-region metrics.","marker":"[112]"}],"fun_headline_variants":["NeRF fails point to mirrors","Mirror detection from NeRF inconsistencies","Find mirrors by NeRF's weak spots","Where NeRF blurs, mirrors appear","Detect mirrors from NeRF artifacts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The detection step assumes mirror regions make a standard NeRF render structurally wrong but depth-confident; if the mirror's reflection can be absorbed as ordinary fake geometry because the space behind it was never photographed, the score never fires and no primitive is found.","fun_headline_variants_meta":{"raw":{"variants":["NeRF fails point to mirrors","Mirror detection from NeRF inconsistencies","Find mirrors by NeRF's weak spots","Where NeRF blurs, mirrors appear","Detect mirrors from NeRF artifacts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000374,"raw_usage":{"total_tokens":2039,"prompt_tokens":1027,"completion_tokens":1012,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":836}},"tokens_in":643,"tokens_out":1012,"duration_ms":8112,"temperature":1.0,"reasoning_tokens":836,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:41:25.089732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture a multi-view scene with a mirror whose reflected content lies entirely outside the training views (for example, a mirror on a wall facing a closed corridor), run the initial NeRF with the depth reprojection loss, and check whether any mirror pixels receive a high score $s(r)$. The paper's own proxy-geometry discussion predicts they will not, which would show the automatic detection is not general across capture configurations.","supporting_citations":[{"cited_title":"Srinivasan, Matthew Tancik, Jonathan T","cited_arxiv_id":null,"evidence_quote":"Defines the standard NeRF whose initial training produces the photometric inconsistency signal and whose quality serves as a baseline."},{"cited_title":"Sparf: Neural radiance fields from sparse and noisy poses","cited_arxiv_id":null,"evidence_quote":"Provides the depth reprojection loss used in the initial stage to make geometry plausible enough for mirror detection."},{"cited_title":"Image quality assessment: from error visibility to structural similarity","cited_arxiv_id":null,"evidence_quote":"Defines the SSIM metric that forms the structural dissimilarity component of the per-ray mirror score."},{"cited_title":"Least squares quantization in pcm","cited_arxiv_id":null,"evidence_quote":"Provides k-means clustering, used to segment the candidate point cloud into one region per mirror primitive."},{"cited_title":"Multi-space neural radiance fields","cited_arxiv_id":null,"evidence_quote":"Serves as the main annotation-free multi-space baseline that the paper compares against and outperforms in mirror-region metrics."}],"review_version":1}