{"id":"22c8c290-6bb2-4f15-bf94-dc2b607134d0","arxiv_id":"2505.11516","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SELECT selects annotation voxels using feature-variance ranking, Monte Carlo dropout uncertainty, and a class-balance entropy criterion, and reports mIoU gains over prior active learning baselines on three LiDAR benchmarks.","lead":"SELECT is a three-stage active learning pipeline that chooses which voxels of a LiDAR scan to send to human annotators. The paper claims it selects informative and class-balanced regions, improving segmentation accuracy on SemanticPOSS, SemanticKITTI, and nuScenes under a limited labeling budget.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage 3 balances classes using the model's own predicted labels (Eqs. 8–10); when rare classes receive zero predicted mass, the balancing signal is self-confirming, so the claimed SMPCB benefit is not yet established.","rationale":"I agree with the reader's weakest_assumption. The concern is load-bearing because the empirical gain is the central claim and Stage 3 is the only component aimed at class imbalance; its objective is computed from predicted labels, which are least reliable for the rare classes it is supposed to recover. The additive Stage 1 and Lambda3=1 issues are real framing overstatements but do not by themselves falsify the empirical trend; the predicted-label dependency is a correctness risk in the mechanism. The concrete oracle test would distinguish a robust balancing effect from a self-confirming artifact. The reader's CONDITIONAL verdict already captures the need for this evidence, so no change is needed.","tokens_in":20847,"tokens_out":10298,"duration_ms":114658,"concrete_test":"On SemanticPOSS with MinkNet, rerun the full SELECT pipeline but replace the predicted counts N_c^Vj in Eq. 8 with ground-truth counts for candidate voxels (oracle; evaluate on a held-out split or use labels from a later annotation round). Compare final mIoU and per-class rare-class mIoU against the reported full SELECT (47.77) and Stages 1+2 only (43.28). If the oracle variant does not improve rare-class mIoU over the predicted-label variant, the predicted-label dependency is not the bottleneck and the concern does not land; if oracle improves substantially, Stage 3's current balancing signal is limited by model miscalibration and the claimed class-balance benefit is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is empirical: SELECT's three-stage selection yields consistently higher mIoU than prior AL methods. The load-bearing condition is that the Stage 3 class-balance criterion actually improves selection, and that its benefit is not an artifact of the model's own errors. In Stage 3, Eq. 8 uses N_c^Vj, the number of points in candidate voxel Vj predicted to belong to class c, to estimate the class proportions Λ_j,c; Eqs. 9–10 then select the voxel maximizing entropy of the softmax-normalized proportions. If the current model assigns zero probability mass to a rare class, a voxel whose true label is that class contributes zero count to N_c^Vj, so the entropy objective cannot prefer it. This is not hypothetical: the paper's own ablation (Table III) shows that Stages 1+2 alone give 0 mIoU on garbage-can and cone/stone on SemanticPOSS, exactly the regime where predicted labels for rare classes are absent. The full pipeline's Stage 3 adds +4.49 mIoU over Stages 1+2, so the claimed rare-class improvement is attributed to a criterion driven by these same unreliable predictions. The paper reports no calibration analysis, no sensitivity test for the predicted-label dependence, and no error bars, so the central empirical gain could be brittle or partly coincidental. The correct response is to test the dependency directly, not to assume the mechanism works.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SELECT, a three-stage active learning pipeline for LiDAR semantic segmentation. Stage 1 ('Voxel-Level Submodular Subset Selection') scores voxels by the variance of their averaged feature vectors and claims to select a diverse representative subset. Stage 2 ('Voxel-Level Model Uncertainty Estimation') ranks the surviving voxels using MC-dropout logits. Stage 3 ('Submodular Maximization for Point-Level Class Balancing') selects a final voxel per point cloud using an entropy criterion over predicted class proportions. The authors report consistent mIoU gains over a wide set of baselines on SemanticPOSS, SemanticKITTI, and nuScenes, with headline improvements of +8.17, +5.06, and +5.05 mIoU over Annotator, respectively, under a 6000-point budget. The central claim is that SELECT constitutes a unified submodular framework that is simultaneously scalable, uncertainty-aware, and class-balanced.","tokens_in":21126,"tokens_out":6897,"duration_ms":73065,"significance":"If the empirical results hold, the practical significance is real: reducing annotation cost in outdoor LiDAR semantic segmentation is an important problem, and the paper compares against many recent AL baselines on three standard benchmarks. The per-class tables and the stage ablations are useful evidence that rare-class performance can be improved by selection strategies. However, the significance is currently undercut by three issues: the 'submodular' Stage 1 objective is additive and reduces to variance ranking; Stage 3 with Lambda3=1 is a single entropy evaluation rather than a submodular maximization; and the class-balance stage depends on predicted labels in exactly the regime where the paper itself shows rare-class predictions are zero. The absence of error bars and code further weakens the support for the headline numerical claims. The manuscript would be a valuable practical contribution after these claims are corrected and the corresponding experiments are added, but in its present form the technical contribution is overstated.","major_comments":[{"comment":"Stage 1 is not a submodular subset selection in any meaningful sense. The objective in Eq. (5) is f(S) = sum over V_j in S of g(sigma(f_{V_j})), so the marginal gain of adding a voxel is g(sigma(f_{V_j})), independent of the current set S; the function is additive, not merely submodular. Under the cardinality constraint the maximizer is simply the Lambda1 voxels with the largest sigma values, and no diversity, redundancy reduction, or pairwise interaction is enforced. Moreover, Eq. (4) computes the variance across the D components of the averaged voxel feature, not the spread of point-level features inside the voxel, so the text's claim that the score 'captures dissimilarity between point-level features' is not supported. The authors should either introduce an interaction term that actually encourages diversity or reframe Stage 1 as top-k variance selection and remove the submodularity claim from the paper's central contribution.","section":"IV-A, Eqs. (3)-(5)"},{"comment":"With the experimental setting Lambda3=1, Stage 3 chooses a single voxel per point cloud per active-learning round. The greedy 'submodular maximization' is therefore a single evaluation of H(V_j) for each candidate, not an iterative optimization of a submodular function. The claimed contribution of submodular class balancing is not exercised by the reported protocol. If the authors wish to claim submodular maximization, they should either select multiple voxels per round at Stage 3 or explicitly clarify that submodularity is only a motivation for a one-step entropy heuristic.","section":"IV-C, Eqs. (8)-(10), V-A"},{"comment":"Stage 3 uses the model's predicted labels N_c^{V_j} to estimate the class composition of each candidate voxel. When the segmentation model is miscalibrated and assigns zero probability mass to a rare class, a voxel that actually contains that class contributes zero count to N_c^{V_j}, so the entropy objective cannot prefer it. This is not hypothetical: the paper's own ablation in Table III shows that Stages 1+2 alone produce 0 mIoU on the garbage-can and cone/stone classes on SemanticPOSS, and the full pipeline attributes +4.49 mIoU over Stages 1+2 to Stage 3. The manuscript provides no calibration analysis, no sensitivity test for the predicted-label dependency, and no error bars. The authors should add such an analysis or use a label-free uncertainty-aware class-balance criterion.","section":"IV-C, Eq. (8)"},{"comment":"The annotation budget is specified as 6000 points, but the protocol states that every method selects Nq=1 voxel per point cloud per round. If all training scans are used, the number of selected voxels would be many thousands, and even a handful of voxels per scan would already exceed 6000 points; if only a subset of scans is used, the selection of that subset is not described. This makes the budget comparison and the reported point-level cost difficult to reproduce. Please specify the number of point clouds used per round, how the 6000-point budget is shared across point clouds, and how voxel point counts are counted.","section":"V-A"},{"comment":"All reported mIoU values appear to be from a single run; no standard deviations, seeds, or code are provided. Given that the headline improvements are 5-8 mIoU points and several baselines differ by only 1-2 points, the claim of consistent superiority is not statistically supported. Please provide multiple seeds with error bars for at least the main tables and make code and configuration public.","section":"V-B, Tables I-II"}],"minor_comments":[{"comment":"The abstract contains a typo: 'understating' should be 'understanding', and the comma in 'costly and, requires' should be removed.","section":"Abstract"},{"comment":"The notation P_i = {V_j = {...}_{k=1}^{N_j}}_{j=1}^{N_i} in Section III is malformed and should be rewritten for clarity.","section":"III"},{"comment":"Table II column headers contain OCR artifacts such as 'Ba.ier', 'Fraff.', and 'Sqn', and several class names are inconsistently capitalized; please proofread the tables.","section":"Tables I-II"},{"comment":"The text should state explicitly that the variance in Eq. (4) is over feature dimensions of the mean voxel feature, not over the point features within the voxel, to avoid the misleading wording in the surrounding paragraph.","section":"IV-A, Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the empirical pipeline is plausible and the datasets are standard, but the submodular framing is currently overstated, and the headline gains lack statistical support. I would suggest asking the authors to compare against a simple top-k variance baseline and to report standard deviations over multiple seeds before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this one. First, the empirical core is probably real: a three-stage voxel-centric active learning pipeline for LiDAR semantic segmentation with consistent gains on SemanticPOSS, SemanticKITTI, and nuScenes under a fixed 6000-point budget. Second, the submodular framing is mostly decorative. Stage 1's objective is a sum of per-voxel scores, so greedy selection reduces to top-k ranking by a hand-defined variance score with no diversity coupling; Stage 3 with Lambda3=1 is a single entropy evaluation, not a submodular maximization. Read it as a well-engineered heuristic pipeline with an overstated name.\n\nWhat is actually new: the three-stage combination itself - feature-variance ranking, MC-dropout uncertainty, and a predicted-label class-balance criterion at the voxel level - does not appear in the prior literature, and the point-budget protocol (one voxel per scan per round, budget counted in points rather than voxels) is a fair way to compare voxel-based methods. The experiments are broad: two backbones, three datasets, thirteen baselines including Annotator. The citations cover the right prior work and the self-citations are proportionate. Their own ablation is the most honest section: Stage 2 does the heavy lifting (+14 mIoU over Random versus +5 for Stage 1), and the full pipeline adds another 4.5 points.\n\nThe soft spots are real but addressable. The load-bearing concern is Stage 3's self-referential loop. Equations 8-10 use the model's predicted labels to estimate class proportions, and their own Table III shows that Stages 1+2 alone score 0 mIoU on garbage-can and cone/stone - exactly the rare classes where the model assigns no predicted mass. Stage 3 then takes credit for a +4.49 gain through a criterion driven by the same unreliable predictions. There is no calibration analysis, no sensitivity test for that dependency, and no error bars. There are also smaller mismatches: the abstract's +8.17 mIoU on SemanticPOSS does not match Table I (46.90 vs Annotator's 35.40 is +11.5); the text cites MinkNet per-class numbers that sit in a Supplementary not present in the arXiv version; and Table II's column headers are garbled. None of these are load-bearing enough to reject, but all need fixing.\n\nWho it is for: people working on label-efficient 3D segmentation will get a useful heuristic and a solid comparison table. It deserves a serious referee. Send it out, but require code, error bars, a direct sensitivity test of the predicted-label dependence in Stage 3, and a rewrite that describes Stage 1 as variance ranking and Stage 3 as entropy balancing. The submodular vocabulary should be earned, not borrowed.","headline":"A pragmatic three-stage heuristic for active LiDAR segmentation with real-looking gains, but the submodular framing is overstated and the class-balancing stage leans on the very predictions it is meant to fix.","tokens_in":21700,"tokens_out":8785,"would_cite":false,"duration_ms":80051,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Submodular three-stage selection lifts LiDAR mIoU by up to 8 points","keywords":["active learning","LiDAR semantic segmentation","submodular optimization","voxel-centric selection","class imbalance","Monte Carlo dropout","point cloud annotation","label efficiency"],"falsifier":"Run SELECT on SemanticPOSS with Stage 3's class proportions computed from ground-truth labels instead of model predictions; if the mIoU gain over the predicted-label version is small, the balancing stage is robust, while a large gap would show the stage is driven by the very misclassifications it is meant to fix.","tokens_in":20597,"feed_emoji":"📡","tokens_out":3344,"duration_ms":31878,"temperature":0.7,"pith_summary":"The paper claims that active learning for LiDAR semantic segmentation can be made both scalable and class-aware by selecting annotation units at the voxel level rather than the point level. It proposes SELECT, a three-stage pipeline that first picks a diverse candidate set of voxels with a variance-based submodular score, then ranks them by Monte Carlo dropout uncertainty, and finally chooses the voxel whose predicted class distribution best balances the labeled set. On SemanticPOSS, SemanticKITTI, and nuScenes, the paper reports mIoU gains of +8.17, +5.06, and +5.05 percentage points over the previous state of the art under a 6,000-point labeling budget. If true, this means substantially fewer annotated LiDAR points are needed to reach a given segmentation accuracy in outdoor scenes.","feed_headline":"Submodular three-stage selection lifts LiDAR mIoU by up to 8 points","feed_subtitle":"A voxel-level three-stage active learner beats prior methods on SemanticPOSS, SemanticKITTI, and nuScenes within a 6,000-point budget.","key_machinery":"The central object is the three-stage voxel-centric selection pipeline. Stage 1 uses a variance-based submodular score on averaged point features with a concave gain function, maximizing a submodular objective via greedy selection to avoid pairwise distance computations. Stage 2 computes voxel-level uncertainty by averaging Monte Carlo dropout logits and taking the majority predicted class per voxel. Stage 3 estimates the class distribution of already-selected voxels and each candidate voxel, applies softmax normalization, and greedily selects voxels that maximize the entropy of the resulting class distribution. The diminishing-returns property of submodular functions is what lets the framework encourage diversity and balance while remaining scalable.","core_discovery":"The central claim is that a voxel-centric, unified submodular framework can jointly optimize informativeness, uncertainty, and class balance in active LiDAR semantic segmentation. SELECT works in three coordinated stages: it uses a variance-based submodular function to select a representative set of voxels without pairwise comparisons, aggregates Monte Carlo dropout logits to identify voxels where the model is uncertain, and then applies an entropy-based submodular maximization step to choose voxels that push the labeled point distribution toward class balance. The paper demonstrates this on three outdoor benchmarks with two backbones, and reports that the full three-stage pipeline outperforms all prior active learning baselines, with particularly strong per-class gains on rare classes such as pedestrians, riders, and cones.","pith_inferences":["Because Stage 3 uses the model's predicted labels to estimate class proportions, a calibration failure on rare classes would bias the balancing step; a testable extension is to weight pseudo-labels by confidence or to compare with oracle class counts.","The paper uses voxel size 0.25 for selection and 0.05 for training, so the sensitivity of the full pipeline to selection-voxel granularity is a natural next experiment that could trade rare-class recall against computational cost.","The observation that SELECT selects about 9% multi-class voxels versus 5% for the prior baseline suggests that boundary and rare-class voxels are only a small fraction of the pool, implying a possible ceiling on gains achievable from uncertainty alone.","A direct test of the framework's generality would be to apply the same three stages to indoor LiDAR or dense point cloud segmentation, where voxel label homogeneity is weaker."],"forward_implications":["Annotation budgets can be spent at voxel granularity, reducing human labeling effort while preserving a point-level budget control mechanism.","Rare classes such as pedestrian, rider, and cone receive enough selected points to reach nonzero mIoU where many baselines score near zero.","The feature-based submodular selection scales linearly with candidate voxels, avoiding clustering methods that require pairwise distance matrices.","Consistent gains across SemanticPOSS, SemanticKITTI, and nuScenes with two backbones suggest the pipeline transfers across sensor configurations and class taxonomies."],"supporting_citations":[{"why":"Provides the voxel-centric Annotator baseline and the voxel-level selection paradigm that SELECT extends.","marker":"[84]"},{"why":"Supplies Monte Carlo dropout as the uncertainty estimation technique used in Stage 2.","marker":"[66]"},{"why":"Establishes the submodularity background and the property that entropy is submodular, supporting the Stage 3 objective.","marker":"[71]"},{"why":"Provides the greedy maximization result that justifies the efficient selection procedures.","marker":"[68]"},{"why":"Supplies the SPVCNN backbone used for the SemanticPOSS evaluation.","marker":"[14]"},{"why":"Supplies the MinkNet backbone used for SemanticKITTI and nuScenes evaluations.","marker":"[12]"},{"why":"Provides the SemanticPOSS benchmark and its imbalanced label distribution.","marker":"[20]"},{"why":"Provides the SemanticKITTI benchmark used for evaluation.","marker":"[4]"},{"why":"Provides the nuScenes benchmark used for evaluation.","marker":"[1]"}],"fun_headline_variants":["Submodular active learning lifts LiDAR mIoU by 8 points","Voxel-centric submodular selection beats prior LiDAR AL","Scalable submodular method tackles LiDAR class imbalance","Three-stage submodular AL boosts rare-class LiDAR accuracy","SELECT: submodular active learning without pairwise comparisons"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Stage 3 assumes the segmentation model's predicted labels are accurate enough to estimate the class distribution of both already-selected and candidate voxels, even for the rare classes the active learning loop exists to improve.","fun_headline_variants_meta":{"raw":{"variants":["Submodular active learning lifts LiDAR mIoU by 8 points","Voxel-centric submodular selection beats prior LiDAR AL","Scalable submodular method tackles LiDAR class imbalance","Three-stage submodular AL boosts rare-class LiDAR accuracy","SELECT: submodular active learning without pairwise comparisons"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001127,"raw_usage":{"total_tokens":4689,"prompt_tokens":954,"completion_tokens":3735,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":3648}},"tokens_in":570,"tokens_out":3735,"duration_ms":27560,"temperature":1.0,"reasoning_tokens":3648,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:41:02.786730+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SELECT on SemanticPOSS with Stage 3's class proportions computed from ground-truth labels instead of model predictions; if the mIoU gain over the predicted-label version is small, the balancing stage is robust, while a large gap would show the stage is driven by the very misclassifications it is meant to fix.","supporting_citations":[{"cited_title":"Annotator: A generic active learning baseline for LiDAR semantic segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the voxel-centric Annotator baseline and the voxel-level selection paradigm that SELECT extends."},{"cited_title":"Uncertainty in deep learning,","cited_arxiv_id":null,"evidence_quote":"Supplies Monte Carlo dropout as the uncertainty estimation technique used in Stage 2."},{"cited_title":"An analysis of approximations for maximizing submodular set functions—I,","cited_arxiv_id":null,"evidence_quote":"Provides the greedy maximization result that justifies the efficient selection procedures."},{"cited_title":"SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances","cited_arxiv_id":"2002.09147","evidence_quote":"Provides the SemanticPOSS benchmark and its imbalanced label distribution."}],"review_version":1}