{"id":"48f72065-ca4f-4b29-a04b-31fb2f43e23c","arxiv_id":"2505.11934","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"iSegMan enables training-free interactive 3D Gaussian segmentation and manipulation from 2D clicks by combining epipolar click propagation with SAM-based visibility voting, reaching 92.4 mIoU on SPIn-NeRF.","lead":"This paper presents iSegMan, a system that lets users click on a 2D image to select and edit objects in a 3D Gaussian Splatting scene without retraining per scene. It combines epipolar-guided click propagation, SAM masks, and a visibility-based voting step to isolate the target region, then applies functions like color change, scaling, removal, or text-driven editing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"As written, Eq. 7 cannot select ordinary objects: its normalization by h·w·K makes Ψ_j ≈ object footprint fraction, far below the stated 0.8 threshold.","rationale":"I read the paper in good faith: the central claim is that iSegMan achieves precise, training-free 3D region control through EIP and VGV, and that this is reflected in strong quantitative segmentation results. The reader's weakest assumption was that the fixed threshold τ=0.8 may not be robust across scenes. My concern is adjacent but stronger: the selection rule as written is not merely sensitive to τ, it is arithmetically implausible. With Eq. 7's denominator h·w·K, Ψ_j is the average mask-weighted visibility over all pixels and views. For any object that does not occupy nearly the whole image, this average is small, so a fixed threshold of 0.8 would select essentially nothing. The paper does not report a per-candidate normalization, nor does it provide code to resolve the ambiguity. If Eq. 7 is a typo and the implementation uses a different normalization, then the paper's central equation misrepresents the method and the reported numbers need to be recomputed under the correct formula. If Eq. 7 is accurate, the central claim fails. This is a correctness risk in the core VGV component, not a peripheral tuning issue. I therefore keep the reader's conditional verdict, but the condition should include resolving this normalization inconsistency, not just adding threshold sensitivity analysis. I do not agree fully with the reader's weakest assumption because the threshold problem is secondary to the definition of the vote itself.","tokens_in":16514,"tokens_out":5969,"duration_ms":68980,"concrete_test":"Construct or reuse a SPIn-NeRF scene with a target object occupying roughly 10–20% of each rendered view, supply perfect ground-truth 2D masks, and compute Ψ_j for every object Gaussian using Eq. 7 exactly as written, then report the maximum. If max Ψ_j < 0.8, the stated threshold is incompatible with the stated normalization. If the released code instead divides by each candidate's total visible pixel mass, the paper must correct Eq. 7 and re-report Tables 2 and 3 under the corrected definition.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing problem is not that τ=0.8 is untuned; it is that the vote quantity defined in Eq. 7 is inconsistent with that threshold. Eq. 7 defines Ψ_j = (1/(h·w·K)) Σ_i Σ_k τ_i[k]·Υ_{i,j}, where Υ_{i,j} is the alpha-blended visibility of Gaussian j at pixel i. For a Gaussian whose splat covers A pixels in each of K views and is perfectly inside the SAM masks, the numerator is roughly K·A, giving Ψ_j ≈ A/(h·w). A target occupying 10% of each view yields Ψ_j ≈ 0.1, far below the 'predetermined threshold of normalized votes is set to 0.8' stated in Appendix 1.2. Only an object filling about 80% of every training view could be selected. Thus either Eq. 7 is missing a per-candidate normalization, such as dividing by the candidate's total visible pixel mass, or the threshold is applied to a different quantity than the one displayed. This is an internal inconsistency in the core VGV mechanism, and it puts the reported 92.4 mIoU / 99.1 mAcc numbers on SPIn-NeRF out of reach of the described selection rule: the rule as written would reject almost all target Gaussians in typical scenes.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents iSegMan, an interactive framework for segmenting and manipulating 3D Gaussian Splatting scenes from 2D clicks. It proposes Epipolar-guided Interaction Propagation (EIP) to propagate clicks across views, Visibility-based Gaussian Voting (VGV) to lift 2D SAM masks to 3D Gaussians without scene-specific training, an Iterative Inspection Mechanism (IIM) to discard unreliable masks, and a manipulation toolbox (semantic editing, colorize, scaling, copy-paste, combination, removal). The authors report mIoU/mAcc of 92.4/99.1 on SPIn-NeRF and 92.0/98.4 on NVOS, qualitative and quantitative comparisons against several prior methods, and ablations of EIP, IIM, and the feature extractor.","tokens_in":16757,"tokens_out":6907,"duration_ms":66354,"significance":"If the reported behavior is reproducible, the framework offers a practical, training-free route from 2D user interactions to 3DGS region control. The strengths of the paper are its clearly specified pipeline, the use of standard benchmarks, the robustness analysis over view sampling rates and view order, and the ablations showing the contributions of the epipolar constraint and the iterative inspection mechanism. However, the central VGV voting rule as described in Eq. (7) is inconsistent with the stated threshold, and the timing comparisons in Tables 2–3 contradict the \"optimal execution speed\" claim. Because these issues affect the core contribution, the significance can only be assessed after they are resolved.","major_comments":[{"comment":"Equation (7) defines Ψ_j as the sum of α-weighted votes normalized by h·w·K, i.e., the fraction of all pixel-view pairs weighted by visibility. For a target whose splats cover A pixels per view on average, the numerator is roughly K·A, so Ψ_j ≈ A/(h·w). A typical object occupying 10% of each view gives Ψ_j ≈ 0.1, far below the 0.8 threshold stated in Appendix 1.2. With the rule as written, only objects covering roughly 80% of every training view could be selected, which is inconsistent with the high mIoU reported on SPIn-NeRF and NVOS, where targets have small footprints. Please correct the normalization (e.g., divide by Σ_i Σ_k Υ_{i,j}) or state explicitly that the threshold applies to a different quantity, and verify the reported numbers with the corrected rule.","section":"Section 3.2, Eq. (7); Appendix 1.2"},{"comment":"In Tables 2 and 3, SAGA reports a per-interaction segmentation time of 10 ms, while iSegMan reports 6 s and 4 s, respectively. The manuscript's claims of \"optimal execution speed\" (Abstract) and \"optimal balance of performance and execution speed\" (Introduction) are therefore unsupported unless they refer to total preprocessing time rather than interactive latency. Please qualify the efficiency claim and clarify which quantity is being optimized.","section":"Section 4.3, Tables 2 and 3; Abstract and Introduction"},{"comment":"Click-Gaussian [10] is cited in Sec. 2.2 as a SAM-based interactive 3DGS segmentation method, but it is absent from the quantitative comparisons in Tables 2 and 3. Because Click-Gaussian directly targets the same task (interactive segmentation of 3D Gaussians from 2D clicks), omitting it weakens the claim of state-of-the-art performance. Please add this baseline or justify its omission.","section":"Section 2.2; Tables 2 and 3"},{"comment":"The vote threshold τ = 0.8 is set by hand in Appendix 1.2 and no sensitivity analysis is reported. Even after correcting the normalization issue in Eq. (7), the threshold is a load-bearing hyperparameter because it determines which Gaussians are selected. Please report mIoU/mAcc for a range of thresholds on at least SPIn-NeRF and demonstrate that the reported results are not an artifact of a single tuned value.","section":"Appendix 1.2; Section 4.4"}],"minor_comments":[{"comment":"The camera pose for the new viewpoint is written as π_˜v = R_˜v[R_˜v|t_˜v], where the same symbol R_˜v is used for both the projection matrix and the extrinsic rotation; Eq. (4) instead uses K_˜v. Please use consistent notation, e.g., Π_˜v = K_˜v[R_˜v|t_˜v].","section":"Section 3.1, text before Eq. (4)"},{"comment":"The update rule in Eq. (9) mixes an L1 loss and a perceptual distance D(·,·) without specifying their relative weights or the exact form of D. Please provide the full loss definition to make the optimization reproducible.","section":"Section 3.3, Eq. (9)"},{"comment":"The correspondence between the original and edited views is not always visually evident; adding view indices or arranging side-by-side comparisons of the same viewpoint would make the qualitative editing results easier to verify.","section":"Figures 4 and 5"},{"comment":"The table captions use 'IIM' without expanding the acronym; please define it in the caption or in the main text to keep the appendix self-contained.","section":"Appendix 3, Tables B and C"}],"recommendation":"major_revision","confidential_remarks":"The inconsistency in Eq. (7) is serious enough that I would ask the authors to provide the actual voting normalization used in their implementation, ideally with code or a precise pseudocode description. If the reported numbers were obtained with a different rule than the one in the paper, the manuscript needs a substantive rewrite of Sec. 3.2 and re-validation. The omission of Click-Gaussian, a directly relevant baseline, is also worth probing in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the takeaway: this is a useful engineering contribution, but the core VGV selection rule is internally inconsistent as written. Eq. 7 defines the normalized vote as (1/(h·w·K)) Σ τ·Υ. For a Gaussian that is visible in, say, 10% of the pixels across views and fully covered by the SAM masks, the numerator is about K·A, so Ψ_j ≈ A/(h·w) ≈ 0.1. The appendix says the threshold is 0.8. That means the described rule would select only objects filling roughly 80% of every view, which is not the setting of the reported experiments. Either Eq. 7 is missing a per-candidate normalization (for example, dividing by the candidate's total visible pixel mass) or the threshold is applied to a different quantity. This is not a side detail: it puts the 92.4 mIoU / 99.1 mAcc results in doubt because the mechanism as stated cannot produce them.\n\nCredit where it's due: EIP is a clean and cheap way to propagate 2D clicks using epipolar lines plus DINO matching, and the sampling-rate robustness analysis (Table 4) is a nice addition. The manipulation toolbox is practical, the qualitative results are visually convincing, and the ablations for epipolar constraint, IIM, and feature extractor are more than most systems papers include. The citation pattern is largely fine, though Click-Gaussian is cited and then not compared, which is a real omission for a training-free 3DGS segmentation method.\n\nOther soft spots are smaller. The \"optimal execution speed\" claim is an overstatement: SAGA's per-interaction segmentation is 10ms versus 4–6s here, even if SAGA needs an hour of feature training. The fixed τ=0.8 without sensitivity analysis is minor compared with the Eq. 7 problem, but once the normalization is fixed, the authors need to show that the threshold is stable across scenes.\n\nThe bottom line: the system idea is worth refereeing and the paper should go to peer review, but the authors must reconcile Eq. 7 with the threshold and add the Click-Gaussian comparison before any acceptance. If I were an editor, I would send it out with a note to reviewers to check the VGV math carefully.","headline":"The system is practical, but the VGV voting equation as written contradicts the stated 0.8 threshold and would reject almost every ordinary object; that must be fixed before the numbers are credible.","tokens_in":17262,"tokens_out":5044,"would_cite":false,"duration_ms":55189,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"iSegMan claims that a 2D click can select a precise 3D region of a Gaussian-splat scene and drive editing operations, with no scene-specific training.","keywords":["3D Gaussian Splatting","interactive segmentation","3D scene manipulation","epipolar geometry","visibility-based voting","region control","text-guided editing","user interaction"],"falsifier":"Sweep the vote cutoff from about 0.5 to 0.95 separately for each scene on the SPIn-NeRF and NVOS benchmarks, computing mean intersection-over-union at each value; if the per-scene cutoff that maximizes accuracy moves substantially away from 0.8 and the accuracy gap is larger than a few points, the claim that one fixed threshold governs reliable region control is falsified.","tokens_in":16313,"feed_emoji":"🎯","tokens_out":7760,"duration_ms":65925,"temperature":0.7,"pith_summary":"iSegMan is an interactive system for selecting and manipulating parts of a 3D scene stored as Gaussian splats. The paper claims that a user who supplies clicks (with optional negative clicks) in any 2D view can isolate a precise 3D region and then edit, recolor, rescale, copy, combine, or remove it, with no per-scene training step. The two mechanisms that carry this claim are epipolar-guided propagation of the click to other views and a visibility-based voting game that maps 2D mask votes onto 3D Gaussians. On the SPIn-NeRF and NVOS benchmarks the method reports 92.4/99.1 and 92.0/98.4 mean intersection-over-union/mean accuracy, with segmentation per interaction in 4 to 6 seconds after a short feature-extraction pass. If correct, this makes 3D scene editing interactive rather than a batch optimization tied to one scene.","feed_headline":"Click to segment and edit 3D scenes in seconds","feed_subtitle":"iSegMan maps a 2D click onto 3D Gaussians and reports over 92 percent mIoU without scene-specific training.","key_machinery":"The carrying object is a per-Gaussian vote tally. A 2D pixel in a given view votes for every 3D Gaussian that contributes to its rendered color, with voting power equal to the Gaussian's alpha-blending weight, computed from the splatting opacity and the accumulated transmittance of occluders in front of it; the final vote share of a Gaussian is the average over pixels and views of these weights, restricted to pixels inside the segmenter's mask. The epipolar constraint is the other load-bearing piece: it turns a 2D click into a ray, restricts the matching search in every other view to a line, and makes cross-view propagation robust with a lightweight feature-similarity match. These two components together replace per-scene feature training, and the iterative inspection mechanism filters views in which the target is occluded or out of frame.","core_discovery":"The central claim is that precise 3D region control can be achieved in Gaussian-splat scenes without any scene-specific training by treating region extraction as a voting game between 2D pixels and 3D Gaussians. User clicks in one view are propagated to other views by projecting the click's ray onto epipolar lines and matching features along those lines; the propagated clicks are fed to a promptable segmenter to obtain 2D masks, and each visible Gaussian accumulates votes weighted by its alpha-blending visibility contribution to each masked pixel. Gaussians whose normalized vote totals exceed a threshold are selected as the target region, and an iterative inspection step discards views where the predicted mask does not overlap the currently selected region. The same region control then drives a toolbox of manipulation functions, including text-driven semantic editing, colorization, scaling, copy-and-paste, combination across scenes, and removal.","pith_inferences":["The same visibility-weighted voting scheme should transfer to other explicit radiance representations that support alpha blending; the paper demonstrates it only on Gaussian splats.","A natural testable extension is an adaptive vote threshold per scene or per click, since the paper uses a single fixed threshold of 0.8 and reports no sensitivity analysis.","Epipolar-guided interaction propagation could serve as a general multi-view annotation propagation tool, for example propagating user scribbles across video frames or across views of a scanned scene.","Held-out tests on heavily occluded or thin objects would probe whether visibility-weighted voting sharpens or blurs the boundary between target and occluder."],"forward_implications":["If iSegMan's claim holds, existing 3D Gaussian editing pipelines can gain local region control without a pre-processing training phase.","Because region extraction is tied to alpha-blended visibility, improvements in opacity estimation or splatting quality should transfer directly to better region selection.","Views can be subsampled to 10 percent with only a small reported drop in accuracy, so segmentation speed can be traded against precision without retraining.","The interaction-driven toolbox enables complex edits to be assembled incrementally rather than specified in one prompt.","Segmentation per interaction in 4 to 6 seconds, if reproducible, moves the method close to a live editing loop rather than offline processing."],"supporting_citations":[{"why":"Supplies the 2D segmentation masks that feed the voting game after user clicks are propagated.","marker":"[24]"},{"why":"Defines the Gaussian splatting representation and the alpha-blending opacity formula used to set each pixel's voting power.","marker":"[21]"},{"why":"Provides the self-supervised feature extractor used for feature-affinity matching along epipolar lines.","marker":"[5]"},{"why":"Serves as the text-conditioned image editor used to produce edited views for semantic editing of selected regions.","marker":"[3]"},{"why":"The main training-based interactive segmentation baseline compared on SPIn-NeRF and NVOS.","marker":"[7]"},{"why":"A Gaussian-based segmentation baseline compared for accuracy and execution time.","marker":"[6]"},{"why":"Provides the Instruct-N2N scenes and the comparison baseline for semantic editing.","marker":"[17]"},{"why":"Supplies the SPIn-NeRF dataset and its MVSeg baseline for the segmentation comparison.","marker":"[32]"},{"why":"Supplies the NVOS dataset and the NVOS baseline for the segmentation comparison.","marker":"[37]"}],"fun_headline_variants":["Click-select 3D regions, then edit without retraining","Pixel-Gaussian voting delivers click-based 3D scene editing","No training needed: click to manipulate 3D Gaussians","Interactive 3D editing via epipolar-guided clicks and voting","Click once, edit 3D scenes with a toolbox of functions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system separates target from background by a single fixed cutoff of 0.8 on the normalized vote score, applied the same way to every scene and dataset; if the best cutoff varies by scene, the reported accuracy numbers and the claim of reliable region control would change.","fun_headline_variants_meta":{"raw":{"variants":["Click-select 3D regions, then edit without retraining","Pixel-Gaussian voting delivers click-based 3D scene editing","No training needed: click to manipulate 3D Gaussians","Interactive 3D editing via epipolar-guided clicks and voting","Click once, edit 3D scenes with a toolbox of functions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1768,"prompt_tokens":1021,"completion_tokens":747,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":660}},"tokens_in":637,"tokens_out":747,"duration_ms":6824,"temperature":1.0,"reasoning_tokens":660,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:44:49.697098+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sweep the vote cutoff from about 0.5 to 0.95 separately for each scene on the SPIn-NeRF and NVOS benchmarks, computing mean intersection-over-union at each value; if the per-scene cutoff that maximizes accuracy moves substantially away from 0.8 and the accuracy gap is larger than a few points, the claim that one fixed threshold governs reliable region control is falsified.","supporting_citations":[{"cited_title":"Segment any- thing","cited_arxiv_id":null,"evidence_quote":"Supplies the 2D segmentation masks that feed the voting game after user clicks are propagated."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Defines the Gaussian splatting representation and the alpha-blending opacity formula used to set each pixel's voting power."},{"cited_title":"Emerg- ing properties in self-supervised vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the self-supervised feature extractor used for feature-affinity matching along epipolar lines."},{"cited_title":"In- structpix2pix: Learning to follow image editing instructions","cited_arxiv_id":null,"evidence_quote":"Serves as the text-conditioned image editor used to produce edited views for semantic editing of selected regions."},{"cited_title":"Seg- ment anything in 3d with nerfs","cited_arxiv_id":null,"evidence_quote":"The main training-based interactive segmentation baseline compared on SPIn-NeRF and NVOS."},{"cited_title":"Instruct-nerf2nerf: Edit- ing 3d scenes with instructions","cited_arxiv_id":null,"evidence_quote":"Provides the Instruct-N2N scenes and the comparison baseline for semantic editing."},{"cited_title":"Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance 4 fields","cited_arxiv_id":null,"evidence_quote":"Supplies the SPIn-NeRF dataset and its MVSeg baseline for the segmentation comparison."},{"cited_title":"Neural volumetric ob- ject selection","cited_arxiv_id":null,"evidence_quote":"Supplies the NVOS dataset and the NVOS baseline for the segmentation comparison."}],"review_version":1}