{"id":"77c15863-eba7-4985-8432-3a5aba500457","arxiv_id":"2505.16399","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Sketchy-3DIS trains a 3D instance segmenter with perturbed, imprecise bounding boxes by generating pseudo point labels and refining predictions coarse-to-fine, achieving state-of-the-art results despite noisier supervision.","lead":"This paper presents Sketchy-3DIS, a method for 3D instance segmentation that trains on inaccurate, 'sketchy' 3D bounding boxes instead of precise point-level annotations. It combines an adaptive box-to-point labeler with a coarse-to-fine segmentator and reports state-of-the-art results on indoor scene benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'leading performance under sketchy boxes' claim is not established because S1 sketchy-box comparisons are missing on S3DIS and incomplete on ScanNet; competitors are mostly evaluated only with accurate boxes.","rationale":"The reader's verdict is conditional, and this stress pass should keep it conditional but for a sharper reason: the experimental protocol does not yet compare against the strongest same-setting baselines. I do not think this is internal inconsistency or fraud; the method's ablations and robustness table are coherent, and the ScanNet S1 comparison against Box2Mask and GaPro is encouraging. The missing S1 rows on S3DIS and for BSNet/CIP on ScanNet are a verification gap rather than a demonstrated failure. Because the headline is a comparative performance claim, a comparative claim needs same-setting comparisons; the proposed retraining check is cheap and decisive. This does not change the reader's conditional verdict; it sharpens the condition that must be met before the central claim can be accepted.","tokens_in":12971,"tokens_out":11886,"duration_ms":106405,"concrete_test":"Use the paper's perturbation code (α=5%, β=5%, γ=5°) to retrain the strongest box-supervised baselines under S1: BSNet+SPFormer and CIP-WPIS on ScanNetV2, and BSNet+ISBNet/GaPro+ISBNet on S3DIS Area 5, with the same backbone and training budget as the original papers. Compare AP50 and AP25 on the standard val/Area-5 splits against Sketchy-3DIS's S1 numbers (65.8/83.1 and 64.6/73.9). If any baseline matches or exceeds either metric, the 'leading under sketchy boxes' claim is falsified as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim — state-of-the-art under sketchy bounding boxes on both ScanNetV2 and S3DIS — hinges on showing Sketchy-3DIS beats existing weak-supervision methods in the same S1 setting. That comparison is missing where it matters most. In Table 2 (S3DIS Area 5), every competitor row is evaluated under S0 accurate boxes; the only S1 row is Sketchy-3DIS itself (50.7 AP / 64.6 AP50). In Table 1 (ScanNetV2), S1 rows exist for only Box2Mask and GaPro; the stronger weak baselines BSNet and CIP-WPIS are reported only under S0. This cannot support 'leading performance' under sketchy supervision, because the same perturbation costs GaPro roughly 17 AP50 points on ScanNet (70.4 to 53.5), so one cannot extrapolate from S0 numbers to S1 behavior. If BSNet or CIP-WPIS were retrained under S1, they might remain at or above the reported 65.8 AP50 / 83.1 AP25 (ScanNet) or 64.6 AP50 / 73.9 AP25 (S3DIS). The current comparison understates this possibility by mixing supervision conditions across rows.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Sketchy-3DIS, a weakly supervised 3D instance segmentation method that trains from bounding box annotations perturbed by scaling, translation, and rotation (\"sketchy boxes\"). The framework combines an adaptive box-to-point pseudo labeler, which assigns points in overlapped boxes to instances and filters background points, with a coarse-to-fine instance segmentator supervised by the resulting pseudo labels. Experiments are reported on ScanNetV2 and S3DIS, and the paper claims state-of-the-art performance under sketchy box supervision, even surpassing some fully supervised methods. The code is made available.","tokens_in":13249,"tokens_out":5789,"duration_ms":42575,"significance":"If the claimed results are substantiated, the paper would make a practically valuable contribution by relaxing the requirement for accurate, compact box annotations in weakly supervised 3D instance segmentation, since real annotations are often imprecise. The proposed adaptive point-to-box assignment for overlapping boxes is a plausible and interesting idea, and the study addresses a realistic annotation scenario not covered by prior box-supervised methods. The paper also provides code and evaluates on standard benchmarks. However, as presented, the evaluation does not yet establish the headline claim because the comparison under the sketchy-box setting is incomplete on both datasets, and the abstract overstates the results relative to the tables.","major_comments":[{"comment":"The S3DIS Area 5 comparison does not include any competing weak-supervision method evaluated under S1 sketchy boxes; the only S1 row is Sketchy-3DIS itself (50.7 AP / 64.6 AP50). Therefore the statement that the method achieves leading performance under sketchy bounding box annotations is unsupported for S3DIS. Please add S1 results for Box2Mask, WISGP, GaPro, BSNet, and CIP-WPIS (or justify why they cannot be evaluated) using the same perturbation parameters as the proposed method.","section":"§4.2, Table 2"},{"comment":"The ScanNetV2 S1 comparison is restricted to Box2Mask and GaPro; the stronger weak-supervision baselines BSNet and CIP-WPIS are reported only under S0 accurate boxes. Since GaPro's AP50 drops by roughly 17 points when moving from S0 to S1 (70.4 to 53.5), S0 numbers cannot be used to estimate S1 performance. Please report S1 results for BSNet, CIP-WPIS, and any other available strong baselines, and provide test-set S1 numbers as well, so that the \"leading performance under sketchy boxes\" claim can be evaluated.","section":"§4.2, Table 1"},{"comment":"The claim that the method \"even outperforms several fully supervised methods using sketchy bounding boxes\" is contradicted by the reported tables. Under S1, the proposed method's AP50 (65.8 on ScanNetV2 val, 64.6 on S3DIS) is lower than every fully supervised method listed in Tables 1 and 2. The only case where it exceeds a fully supervised method (ISBNet by +3.3 AP50 on S3DIS) is the S0 row in Table 2. Please correct the claim or explicitly state that the fully supervised comparison is under S0 accurate boxes, not under S1 sketchy boxes.","section":"Abstract and §4.2/§5"},{"comment":"The exact generation of the S1–S4 sketchy boxes is not specified. The text defines alpha=5%, beta=5%, and gamma=5 for the basic scaling, translation, and rotation operations, but does not state how these are combined to produce S2, S3, and S4. Without this definition, the robustness results in Table 3 cannot be reproduced. Please provide the precise combinatorial procedures and parameter values for all four sketchy levels.","section":"§3.2 and Table 3"}],"minor_comments":[{"comment":"There is a typo: \"SKeychy-3DIS\" should be \"Sketchy-3DIS\".","section":"§4.2"},{"comment":"The two Sketchy-3DIS rows are not labeled with the supervision type in the \"Sup.\" column; the S0 row has a blank entry. Please label both rows explicitly as S0 and S1 for clarity.","section":"Table 2"},{"comment":"The ScanNet test row for Sketchy-3DIS (70.1 AP50, 86.6 AP25) is reported without a sup label. Please clarify whether this result was obtained under S0 or S1 sketchy boxes, since the abstract's claim about sketchy boxes depends on this distinction.","section":"Table 1"},{"comment":"The notation e^{-|cB - cp|} does not specify the norm used for the coordinate difference; please define it (e.g., Euclidean distance) and its units.","section":"§3.3, Eq. (3)"},{"comment":"The thresholding operation sigma is not described; please define the threshold value and how it is applied to the masked features.","section":"§3.4, Eq. (6)"},{"comment":"The conclusion states that performance \"degraded severely when the annotated sketchy bounding boxes are immensely inaccurate,\" but the experiments only test S1–S4, which are combinations of the small basic perturbations (5% scaling, 5% translation, 5 degrees rotation). This statement is not supported by the reported results; please either add experiments with larger perturbations or soften the claim.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The central weakness is the incomplete S1 comparison: without S1 numbers for BSNet, CIP-WPIS, and the S3DIS baselines, the claimed state-of-the-art under sketchy boxes is not established. The authors can likely address this by retraining baselines under the S1 perturbation and correcting the overclaim in the abstract. The paper fits the scope of the venue, and the proposed method is plausible, so major revision seems appropriate rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper introduces a new weak supervision setting for 3D instance segmentation: sketchy bounding boxes, which are ground-truth boxes perturbed with scaling, translation, and rotation. The setting is genuinely new relative to the cited prior work, which assumes accurate or slightly loose boxes, and the proposed method (adaptive box-to-point pseudo labeler plus coarse-to-fine segmentator) is clearly described and carefully ablated. The robustness table from accurate to heavily perturbed boxes is a good addition, and the gains over Box2Mask and GaPro under the same S1 perturbation on ScanNetV2 are real (65.8 vs 53.5 AP50 for GaPro).\n\nThe main weakness is that the central claim—state-of-the-art under sketchy boxes on both benchmarks—is not actually established by the tables. On S3DIS, every baseline is evaluated only under accurate boxes (S0); the only S1 row is Sketchy-3DIS itself. On ScanNet, the stronger weak baselines BSNet and CIP-WPIS are also only reported under S0, with only Box2Mask and GaPro given S1 numbers. Since GaPro loses about 17 AP50 points when moving from S0 to S1, you cannot assume the other baselines would stay at their S0 values. It is entirely possible that BSNet or CIP-WPIS, retrained under S1, would match or exceed the reported 65.8 AP50. The paper needs to run those baselines under S1 to support its headline.\n\nA second, smaller but more clear-cut issue: the abstract claims the method \"even outperforms several fully supervised methods\" using sketchy boxes. The tables do not show that. On ScanNet, fully supervised SPFormer reaches 73.9 AP50; Sketchy-3DIS under S1 gets 65.8. On S3DIS, fully supervised ISBNet reaches 65.8 AP50; Sketchy-3DIS under S1 gets 64.6. That claim should be corrected or removed.\n\nA minor point: the sketchy boxes are all synthetic perturbations of GT boxes (5% scaling, 5% translation, 5 degrees rotation). Real annotation noise may be larger and class-correlated, so the robustness might not transfer. There are also no error bars, so we don't know run-to-run variance. These are fixable and do not undermine the core idea.\n\nOverall, this is a solid engineering contribution with a new problem formulation. It deserves a serious referee. The revision should add S1 comparisons for BSNet and CIP-WPIS, fix the abstract, and ideally include a test on a realistic noisy annotation set.","headline":"A worthwhile new setting and a sensible method, but the paper's headline claim about state-of-the-art under sketchy boxes is not yet supported by the comparisons it reports.","tokens_in":13771,"tokens_out":2012,"would_cite":true,"duration_ms":17024,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training on inaccurate 'sketchy' bounding boxes yields state-of-the-art 3D instance segmentation, rivaling fully supervised methods.","keywords":["3D instance segmentation","weakly supervised learning","sketchy bounding boxes","pseudo labeling","point cloud","coarse-to-fine segmentation","ScanNet","S3DIS"],"falsifier":"Collect bounding boxes drawn quickly by human annotators on ScanNet or S3DIS, measure the actual scale, translation, and rotation error distribution, and retrain or re-evaluate Sketchy-3DIS on those annotations; if AP50 falls below the fully supervised baseline or by more than the gap claimed for synthetic sketchy boxes, the central robustness claim is falsified.","tokens_in":12773,"feed_emoji":"📦","tokens_out":8273,"duration_ms":59565,"temperature":0.7,"pith_summary":"Sketchy-3DIS tackles the practical reality that 3D bounding-box annotations are rarely precise: annotators introduce scaling, translation, and rotation errors. The paper claims that a joint box-to-point pseudo labeler and a coarse-to-fine query-based segmentator can turn such inaccurate boxes into high-quality per-point instance labels. On ScanNetV2 and S3DIS, the method reaches state-of-the-art results under this sketchy-box supervision and, in several settings, outperforms fully supervised baselines. The practical payoff is annotation cost: labeling a 3D box takes roughly 1.93 minutes per scene, versus 22.3 minutes for point-level instance labels.","feed_headline":"Sketchy boxes rival full supervision for 3D instance segmentation","feed_subtitle":"Box-to-point pseudo labeling plus coarse-to-fine refinement beats several fully supervised baselines on indoor scans.","key_machinery":"The machinery is a two-module joint-training loop. The adaptive box-to-point pseudo labeler partitions points into background, single-box, and overlap sets; it filters background by the similarity $s_{p,B} = \\cos(f_p, f_B) e^{-|c_B - c_p|}$, and assigns overlap points with an MLP that compares point features to the features of the reliable (non-overlap) parts of each box. The coarse-to-fine instance segmentator uses query-based attention over the whole scene, then over regions inside a predicted box and a contracted 'core' box, in Multi-level Attention Blocks. The two modules are supervised by a cross-entropy pseudo-label loss and an instance segmentation loss, paired by Hungarian matching.","core_discovery":"The central claim is that instance segmentation can be learned from sketchy bounding boxes—boxes perturbed by 5% scaling, 5% translation, and 5 degrees rotation from ground truth—without sacrificing competitive accuracy. The paper shows that existing box-supervised methods lose substantial performance under such perturbations, while Sketchy-3DIS maintains state-of-the-art results by converting the rough boxes into compact pseudo instance labels. The conversion is adaptive: points inside overlapping boxes are assigned by a learned point-to-box similarity, and background points inside otherwise reliable boxes are filtered by a coordinate-plus-feature similarity score. A coarse-to-fine segmentator then refines instances by attending to scene-level, coarse-region, and core-region features, and joint training with the pseudo labeler progressively improves both. On the ScanNetV2 validation split the method reaches 65.8 AP50 with sketchy boxes (68.8 with accurate boxes), and on S3DIS Area 5 it reaches 64.6 AP50 with sketchy boxes, exceeding several fully supervised baselines.","pith_inferences":["The synthetic perturbation model (uniform scaling, translation, rotation) is a proxy for human annotation errors; real 'sketchy' boxes may have correlated or class-dependent errors, so a natural test is to benchmark on manually drawn loose boxes rather than perturbed ground truth.","The method's ability to segment objects missing from the ground truth (e.g., cabinets in Fig. 6) hints it could serve as a noise-tolerant annotator that discovers unlabeled instances, not merely a recognizer of annotated ones.","Because the pseudo labeler resolves overlap only between two boxes, scenes with heavy multi-object overlap (e.g., clutter) may need higher-order assignment; extending the MLP to $k$-box overlaps is a direct testable extension.","The degradation from S1 to S4 (65.8 to 62.5 AP50) suggests there is a perturbation budget; measuring the method's tolerance as a function of $\\alpha$, $\\beta$, $\\gamma$ would let practitioners decide when to invest in more careful box annotation."],"forward_implications":["Weakly supervised 3D instance segmentation can work from loose, inaccurate boxes, reducing annotation effort to roughly the cost of drawing a box per object rather than labeling every point.","Existing box-supervised methods lose a large slice of performance under sketchy boxes, so evaluating robustness to annotation noise becomes a necessary part of the comparison.","Joint training of the pseudo labeler and the segmentator clearly beats a disjoint pipeline (AP rises from 60.4 to 69.1 on S3DIS Area 5), so the two components should be optimized together.","The method's advantage holds on two independent indoor benchmarks, ScanNetV2 and S3DIS, indicating the approach is not tuned to a single dataset.","Even with accurate boxes the method remains competitive with the strongest weakly supervised baselines, so adopting sketchy-box training does not give up the clean-annotation regime."],"supporting_citations":[{"why":"Provides the ScanNetV2 benchmark and the annotation-cost numbers (22.3 min for point-level vs 1.93 min per box) that motivate the setting.","marker":"[6]"},{"why":"Provides the S3DIS benchmark used for the second evaluation and generalization claim.","marker":"[1]"},{"why":"The first bounding-box-supervised 3D instance segmentation method, the baseline that the sketchy-box setting extends.","marker":"[4]"},{"why":"A box-supervised baseline using geometric priors, compared in the main tables.","marker":"[8]"},{"why":"The leading box-supervised pseudo-labeler (Gaussian processes) whose performance drops under sketchy boxes, the main competitive baseline.","marker":"[29]"},{"why":"Recent box-supervised mean-teacher baseline that requires synthesized scenes, compared to show Sketchy-3DIS does not need them.","marker":"[27]"},{"why":"SPFormer codebase and query-based architecture used as the backbone for ScanNetV2 training.","marker":"[36]"},{"why":"ISBNet codebase and instance-aware sampling backbone used for S3DIS training and the fully supervised baseline exceeded.","marker":"[30]"}],"fun_headline_variants":["Sketchy boxes match fully supervised 3D instance segmentation","Rough 3D boxes yield state-of-the-art instance segmentation","Perturbed boxes: a cheap path to state-of-the-art 3D segmentation","Inaccurate boxes, accurate results: 3D instance segmentation","Sketchy boxes: from rough bounds to precise 3D segments"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on two linked premises: synthetic perturbations (5% scale, 5% translation, 5 degrees rotation) faithfully model real-world sketchy boxes, and points that fall inside only one box are clean enough to bootstrap the overlap assignment.","fun_headline_variants_meta":{"raw":{"variants":["Sketchy boxes match fully supervised 3D instance segmentation","Rough 3D boxes yield state-of-the-art instance segmentation","Perturbed boxes: a cheap path to state-of-the-art 3D segmentation","Inaccurate boxes, accurate results: 3D instance segmentation","Sketchy boxes: from rough bounds to precise 3D segments"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001707,"raw_usage":{"total_tokens":6792,"prompt_tokens":1014,"completion_tokens":5778,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":5684}},"tokens_in":630,"tokens_out":5778,"duration_ms":34801,"temperature":1.0,"reasoning_tokens":5684,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:01:01.473874+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect bounding boxes drawn quickly by human annotators on ScanNet or S3DIS, measure the actual scale, translation, and rotation error distribution, and retrain or re-evaluate Sketchy-3DIS on those annotations; if AP50 falls below the fully supervised baseline or by more than the gap claimed for synthetic sketchy boxes, the central robustness claim is falsified.","supporting_citations":[{"cited_title":"ScanNet: Richly- annotated 3D reconstructions of indoor scenes","cited_arxiv_id":null,"evidence_quote":"Provides the ScanNetV2 benchmark and the annotation-cost numbers (22.3 min for point-level vs 1.93 min per box) that motivate the setting."},{"cited_title":"3D semantic parsing of large-scale indoor spaces","cited_arxiv_id":null,"evidence_quote":"Provides the S3DIS benchmark used for the second evaluation and generalization claim."},{"cited_title":"Box2Mask: Weakly supervised 3D semantic instance segmentation using bounding boxes","cited_arxiv_id":null,"evidence_quote":"The first bounding-box-supervised 3D instance segmentation method, the baseline that the sketchy-box setting extends."},{"cited_title":"Weakly-supervised point cloud instance segmentation with geometric priors","cited_arxiv_id":null,"evidence_quote":"A box-supervised baseline using geometric priors, compared in the main tables."},{"cited_title":"GaPro: Box-supervised 3D point cloud instance segmentation using gaussian processes as pseudo labelers","cited_arxiv_id":null,"evidence_quote":"The leading box-supervised pseudo-labeler (Gaussian processes) whose performance drops under sketchy boxes, the main competitive baseline."},{"cited_title":"BSNet: Box- supervised simulation-assisted mean teacher for 3D instance segmentation","cited_arxiv_id":null,"evidence_quote":"Recent box-supervised mean-teacher baseline that requires synthesized scenes, compared to show Sketchy-3DIS does not need them."},{"cited_title":"Superpoint transformer for 3D scene instance segmentation","cited_arxiv_id":null,"evidence_quote":"SPFormer codebase and query-based architecture used as the backbone for ScanNetV2 training."},{"cited_title":"ISBNet: a 3D point cloud instance segmentation network with instance- aware sampling and box-aware dynamic convolution","cited_arxiv_id":null,"evidence_quote":"ISBNet codebase and instance-aware sampling backbone used for S3DIS training and the fully supervised baseline exceeded."}],"review_version":1}