{"id":"9129561d-127d-4314-a4ed-7d0e5748b785","arxiv_id":"2505.00507","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HeAL scores LiDAR frames for labeling by computing classwise KL divergence between detections on original and 180-degree-rotated point clouds, plus distance and point-count corrections.","lead":"This paper introduces HeAL, an active learning method for LiDAR-based 3D object detection that selects training samples by comparing a scene's detections before and after a 180-degree rotation. It reports matching a fully supervised detector's accuracy using roughly a quarter of the labeled samples.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's claim of matching the full-supervised baseline at 24% of samples is unsupported by any reported full-supervised baseline in the results.","rationale":"I reviewed the full text and considered three candidate concerns. First, the rotation-consistency assumption (the reader's weakest_assumption) is plausible but is a standard augmentation-consistency premise; the paper's ablations provide indirect evidence that the selection signal is useful, so this alone is not the most load-bearing issue. Second, Equations 6-8 may not amplify KL divergence for distant boxes exactly as the prose claims, because scaling covariance upward can reduce the KL between two Gaussians with a given center shift; however, this is a correctable implementation detail and does not directly determine the headline result. Third, and most load-bearing: the abstract's headline claim of matching full supervision at 24% of labels is unsupported because no full-supervised baseline appears in Section 4, Figure 3, or Table 1. This is a verifiable omission: if full-supervised PV-RCNN mAP is several points above HeAL's 69.3% at 700 samples, the abstract's claim would be false. The issue is resolvable by adding one number or horizontal line, so conditional acceptance rather than rejection is appropriate; thus I keep the reader's CONDITIONAL verdict unchanged.","tokens_in":11821,"tokens_out":9227,"duration_ms":103336,"concrete_test":"Ask the authors to report the full-supervised PV-RCNN 3D mAP on the KITTI validation split under identical settings (same optimizer, 30 epochs, 40 recall positions, same classwise AP averaging) and to overlay that value as a horizontal line in Figure 3, along with the exact sample count corresponding to '24%'. Then check whether HeAL's mAP at that budget reaches or exceeds the line within one standard deviation. This single addition would confirm or refute the abstract's central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract—'achieves the same mAP as the full-supervised baseline with only 24% of the samples'—is not backed by any reported comparison. Section 4 and Figure 3 evaluate HeAL against Random, Entropy, Monte Carlo, CRB, and KECOR, but no PV-RCNN model trained on the full KITTI training set is plotted or tabulated, and no full-supervised mAP value is given anywhere. The only quantitative anchor in the paper is Table 1, where the best HeAL variant reaches 69.3% total mAP at 700 samples, yet there is no corresponding full-supervised total mAP to compare against. Without this baseline, the paper's primary value proposition (labeling efficiency versus full supervision) cannot be verified from the manuscript. The '24%' figure is also not cleanly tied to the experimental protocol: eight cycles of 100 samples is 800 samples (about 21.6% of 3,712), so the exact budget at which parity is claimed is ambiguous.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HeAL, an active learning strategy for LiDAR-based 3D object detection. HeAL represents a scene's detections as a Gaussian mixture model, computes a KL-divergence between the original point cloud and a 180-degree rotated version, incorporates class-wise probability maps, and applies distance- and point-density-based covariance corrections. Experiments on KITTI and a nuScenes subset use PV-RCNN within the ACTIVE-3D-DET toolbox, comparing HeAL against Random, Entropy, Monte Carlo, CRB, and KECOR. The abstract claims competitive state-of-the-art mAP and parity with a full-supervised baseline at 24% of the training samples, with ablation studies attributing gains to the class-wise and distance corrections.","tokens_in":12023,"tokens_out":4441,"duration_ms":48935,"significance":"If the claims are substantiated, HeAL offers a practical and architecture-agnostic acquisition function for 3D object detection, with the notable strength of improving rare-class performance and being built from standard detection outputs. The use of a public toolbox, multiple baselines, and three-seed averaged plots are positive aspects that support reproducibility. However, the central value proposition—parity with full supervision at a fraction of the data—is not demonstrated in the reported results, and the ablations are selected and evaluated on the same KITTI benchmark. These issues currently limit the significance of the contribution to a promising but under-verified heuristic.","major_comments":[{"comment":"The abstract's claim that HeAL \"achieves the same mAP as the full-supervised baseline with only 24% of the samples\" is not supported by any reported full-supervised baseline. Figure 3 and the surrounding text compare HeAL only against Random, Entropy, Monte Carlo, CRB, and KECOR; no PV-RCNN trained on the full KITTI training set is plotted or tabulated, and no full-supervised total mAP value is given anywhere. The manuscript must include the full-supervised baseline curve or value, state explicitly the sample budget at which parity occurs, and reconcile the 24% figure with the experimental protocol (8 cycles of 100 samples is 800 samples, about 21.6% of 3,712; Table 1 reports results at 700 samples).","section":"Abstract and Section 4 (Figure 3)"},{"comment":"The ablation in Table 1 identifies GMM+Class+Distance as the best variant on KITTI, and Section 4 then reports HeAL's headline improvements over KECOR, CRB, and Random on the same KITTI benchmark. Because the same dataset is used both to select the variant and to report the final comparison, the reported gains may be optimistic due to selection on the evaluation set. Please provide a clear protocol in which the variant is fixed before the final KITTI comparison, or evaluate the selected variant on a separate held-out split, and report the behavior of all ablated variants across active learning cycles.","section":"Section 4.1, Table 1 vs. Section 4"},{"comment":"The distance and point-count corrections are insufficiently specified to be reproducible and their behavior is not fully justified. In Eq. (6), U_d is normalized by the maximum distance within the detections of a single scene; Eq. (7) uses S, described only as \"the total number of points across all bounding boxes,\" without stating whether this is per scene, per class, or global; and Eq. (8) multiplies the covariance by these corrections. The paper should define all symbols, state the grid resolution and epsilon used in the KL computation, and explain why multiplicatively scaling the covariance produces a meaningful uncertainty estimate rather than simply re-weighting the score by scene composition. The current presentation makes it hard to assess how much of HeAL's gain comes from the augmentation-inconsistency signal versus the heuristic corrections.","section":"Section 3.5, Eqs. (6)–(8)"},{"comment":"The nuScenes experimental protocol is inconsistent with the rest of the paper. Section 3.6.1 reports the same sample split (3,712 training, 3,769 validation) for nuScenes as for KITTI, and Figure 4 shows an x-axis extending to 8,000 labeled samples, which exceeds the stated training-set size. Please clarify the actual nuScenes subset size, the active learning budget and cycle structure, and ensure the axis and text are consistent. Without this, the claimed 2% improvement over KECOR on nuScenes cannot be interpreted.","section":"Section 3.6.1 and Figure 4"}],"minor_comments":[{"comment":"There are numerous typographical errors and inconsistent spellings, including \"arbitratry\" (Contributions), \"vainilla\" (Section 4.1), \"missclasification\" (Section 4.1), \"HeAL\" versus \"Heal\" (Section 4), and \"NuScenes\" versus \"Nuscenes\" (captions and text). The manuscript should be carefully proofread.","section":"Throughout"},{"comment":"Notation is inconsistent: Eqs. (6) and (7) define U_k^d and U_k^p, while Eq. (8) uses U_d and U_p without the subscript k. Please align the notation.","section":"Section 3.5, Eq. (8)"},{"comment":"The legend labels \"HeAL(Points)\" and \"HeAL(Distance)\" are not defined in the caption or text. Please clarify that these correspond to the variants in Table 1 and describe exactly which corrections are active in each curve.","section":"Section 4, Figure 4"},{"comment":"The sentence \"In the low-data regime, only KECOR has the same performance as the Random baseline\" is awkward given that KECOR is described as state-of-the-art; please rephrase to state clearly whether KECOR matches or exceeds Random in this regime.","section":"Section 4, first paragraph"},{"comment":"The statement \"HeAL outperforms the concurrent method for all difficulty levels\" is vague; please name the concurrent method (presumably KECOR) and, if possible, show all baselines per difficulty level rather than only HeAL and one comparison method.","section":"Section 4, Figure 5"},{"comment":"The table reports mean mAP values but no standard deviations, despite the text stating that all plots show mean results over three seeds. Please include variance information or state why it is omitted for the ablation.","section":"Section 4.1, Table 1"},{"comment":"The initial labeled set size is never specified; Section 3.1 defines D_L with cardinality m, but no value is given for the experiments. This is needed to reproduce the active learning curves.","section":"Section 3.6.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a publishable core idea, but the abstract's headline claim about full-supervised parity is unverifiable from the reported results, and the ablations appear to be selected and evaluated on the same benchmark. I would recommend requiring the authors to add the full-supervised baseline curve or to remove the parity claim, and to address the model-selection issue, before this paper can be accepted. The nuScenes reporting inconsistency should also be resolved, as it currently undermines the cross-dataset claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the acquisition function is new and reasonable, but the paper's main claim—full-supervision parity at 24% of the data—is not supported by anything in the results section. If you read it for the method, there is a solid core; if you read it for the headline, you'll be disappointed.\n\nYou already know the ingredients: consistency under 180° rotation, classwise GMMs over detections, and distance/point-count scaling. The specific combination is new, and it has two virtues that matter in practice: it is architecture-agnostic (works off ordinary detection outputs, no auxiliary network) and each component is explainable. The ablation is informative: classwise GMM gives a real bump, distance helps, point-count hurts; that's a useful negative result. The KITTI comparison against KECOR, CRB, Entropy, and Random is a fair range of baselines, and the NuScenes result is a genuine cross-dataset attempt, even if underreported.\n\nThe main problem is verification of the parity claim. The full-supervised mAP is never reported, neither in a table nor as a horizontal line. Figure 3 only plots AL strategies. The 24% is also not tied to the protocol: eight cycles of 100 samples plus an unspecified initial set doesn't obviously equal 24% of 3,712, and the best ablation number (69.3% at 700 samples) has no full-supervised counterpart. That has to be fixed before the paper's value proposition can be taken seriously.\n\nSecond, there are small reproducibility gaps. Eq. 6 defines U_d with distance in the numerator and then Eq. 8 multiplies the covariance by U_d; the direction is consistent with \"far objects get wider Gaussians\", but the text should say so. The grid resolution, sigma_scale, epsilon, and initial labeled set size aren't specified, and code is promised only after acceptance.\n\nThird, mild selection bias: the best correction combination is chosen on KITTI and then reported on KITTI. That's not a fatal flaw, but it should be flagged; the NuScenes figure seems to use the distance and point-only variants, not the full best method, so the cross-dataset claim is weaker than the text implies.\n\nOverall: the central argument—heuristics can improve AL for 3D detection—holds up at a qualitative level. The method deserves a serious referee, but not as is.\n\nRecommendation: send to peer review; ask for the full-supervised curve, exact budget breakdown, implementation details, and a same-protocol NuScenes comparison of the best variant. I'd welcome that revision.","headline":"A plausible, architecture-agnostic AL method whose main headline claim is not backed by the reported experiments.","tokens_in":12554,"tokens_out":3587,"would_cite":true,"duration_ms":38431,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HeAL ranks LiDAR scenes by rotation-consistency and distance heuristics, reaching fully supervised 3D detection accuracy with 24% of labels.","keywords":["active learning","3D object detection","LiDAR","Gaussian mixture model","KL divergence","uncertainty estimation","KITTI benchmark","point cloud"],"falsifier":"On a held-out KITTI split, label the 100 scenes HeAL ranks highest and the 100 it ranks lowest, retrain both from the same starting model, and compare mAP gains: if the low-ranked set improves the detector as much as or more than the high-ranked set over several cycles, then rotation consistency is not the source of the improvement.","tokens_in":11594,"feed_emoji":"🚗","tokens_out":8121,"duration_ms":78070,"temperature":0.7,"pith_summary":"The paper's aim is to make active learning for LiDAR 3D object detection practical: decide which unlabeled scenes should be sent to a human annotator. HeAL scores each scene by how much the detector's output changes under a 180-degree rotation, then weights that change with object distance and point count. The claim is that this score identifies the most informative samples: on KITTI, training with HeAL-selected samples reaches the same 3D mean average precision as a fully supervised model while using only 24% of the labeled data, and after eight selection cycles it outperforms the KECOR, CRB, and random baselines by 3%, 4.9%, and 4.2% mAP respectively. This would matter because LiDAR labeling is expensive and existing methods plateau in the higher-data regime.","feed_headline":"24% of labels match fully supervised 3D detection","feed_subtitle":"Rotation-consistency, distance, and point-density heuristics pick the LiDAR scenes that teach a 3D detector most.","key_machinery":"The central device is a classwise Gaussian mixture model (GMM) built from ordinary detector outputs. Each predicted 3D bounding box becomes a Gaussian blob centered at the box, with covariance taken from its dimensions; a detector's whole scene is thus a sum of blobs per class. The score is the KL divergence between the original scene's GMM and the GMM after a 180-degree rotation that is rotated back into alignment, averaged over classes. Distance and point-count corrections multiply the Gaussian covariance so faraway or sparse objects raise the uncertainty. Because the GMM is built from standard outputs, the mechanism is architecture-agnostic.","core_discovery":"The central discovery is that localization can drive active learning for 3D detectors if the scene's predicted bounding boxes are turned into a classwise Gaussian mixture model and compared, via KL divergence, between the original point cloud and one rotated 180 degrees around the vertical axis. Two heuristic corrections scale the covariance of each Gaussian with the object's distance from the sensor or with the number of LiDAR points inside the box, so that distant and sparse objects contribute more uncertainty. Averaging the classwise KL divergences gives the HeAL score, and the 100 highest-scoring scenes are labeled each round. Ablations show that separating classes into their own probability maps and adding the distance correction produce most of the gain, while the point-count correction alone harms performance.","pith_inferences":["If the mechanism is genuinely about detector instability under label-preserving changes, other perturbations, such as small translations, dropped sensor lines, or temporal shifts, should produce the same ranking; this is testable without new labels.","The distance weighting should transfer best to datasets with long-range detections; a natural extension is to normalize distance by class-specific distributions instead of a global maximum.","The point-count failure suggests occlusion, not sparsity, is the harder failure mode, so an occlusion-aware heuristic may beat both corrections.","The 24% matching point depends on the budget's stopping rule; an economic claim about labeling cost would need AP curves at every budget level for several seeds, which the paper provides for KITTI."],"forward_implications":["If the 24% result transfers, an autonomous-driving team can train a LiDAR detector to full-data accuracy while annotating roughly one scene in four.","The classwise design should put labeling budget on underrepresented classes; HeAL reports clear AP gains for pedestrians and cyclists while the car class stays flat.","Because HeAL uses only standard detection outputs, it can be added to existing pipelines without changing the model or training an extra proxy network.","The paper's suggested split, KECOR for the low-data regime and HeAL later, implies that a fixed budget is best spent with two acquisition functions rather than one.","The ablation result that point-count correction alone hurts means point sparsity should not be used as a standalone uncertainty signal."],"supporting_citations":[{"why":"Defines KECOR, the information-theoretic baseline HeAL is compared against and proposed as a low-data-regime partner.","marker":"[24]"},{"why":"Supplies the CRB baseline and the experimental harness used for all method comparisons.","marker":"[25]"},{"why":"Provides the PV-RCNN detector backbone and the KITTI evaluation protocol with 40 recall positions.","marker":"[31]"},{"why":"Provides the KITTI dataset on which the main training and evaluation are carried out.","marker":"[14]"},{"why":"Supplies the 180-degree rotation augmentation and the heatmap-as-GMM uncertainty idea that HeAL builds on.","marker":"[18]"},{"why":"Establishes rotation-based consistency for active learning in detection, grounding the choice of augmentation.","marker":"[27]"}],"fun_headline_variants":["HeAL3D: 24% labels, same mAP as fully supervised","24% labels match full supervision with HeAL3D heuristics","HeAL3D: distance and point-count heuristics cut labels to 24%","Classwise KL divergence + heuristics slash labels to 24%","Rotation-consistency heuristics pick 24% labels for full 3D mAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's load-bearing premise is that a 180-degree rotation does not change how hard a scene is to detect, so any shift in the detector's output under rotation is a faithful signal of which samples deserve labeling.","fun_headline_variants_meta":{"raw":{"variants":["HeAL3D: 24% labels, same mAP as fully supervised","24% labels match full supervision with HeAL3D heuristics","HeAL3D: distance and point-count heuristics cut labels to 24%","Classwise KL divergence + heuristics slash labels to 24%","Rotation-consistency heuristics pick 24% labels for full 3D mAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002293,"raw_usage":{"total_tokens":8818,"prompt_tokens":884,"completion_tokens":7934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":7831}},"tokens_in":500,"tokens_out":7934,"duration_ms":45694,"temperature":1.0,"reasoning_tokens":7831,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:40:33.224047+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out KITTI split, label the 100 scenes HeAL ranks highest and the 100 it ranks lowest, retrain both from the same starting model, and compare mAP gains: if the low-ranked set improves the detector as much as or more than the high-ranked set over several cycles, then rotation consistency is not the source of the improvement.","supporting_citations":[{"cited_title":"Kecor: Kernel Coding Rate Maximization for Active 3D Object Detection","cited_arxiv_id":null,"evidence_quote":"Defines KECOR, the information-theoretic baseline HeAL is compared against and proposed as a low-data-regime partner."},{"cited_title":"Exploring active 3d object de- tection from a generalization perspective","cited_arxiv_id":null,"evidence_quote":"Supplies the CRB baseline and the experimental harness used for all method comparisons."},{"cited_title":"Pv-rcnn: Point- voxel feature set abstraction for 3d object detection","cited_arxiv_id":null,"evidence_quote":"Provides the PV-RCNN detector backbone and the KITTI evaluation protocol with 40 recall positions."},{"cited_title":"Are we ready for autonomous driving? the kitti vision benchmark suite","cited_arxiv_id":null,"evidence_quote":"Provides the KITTI dataset on which the main training and evaluation are carried out."},{"cited_title":"Efficient active learning strategies for monocular 3d object detection","cited_arxiv_id":null,"evidence_quote":"Supplies the 180-degree rotation augmentation and the heatmap-as-GMM uncertainty idea that HeAL builds on."},{"cited_title":"Camera-lidar inconsistency analysis for active learning in object detection","cited_arxiv_id":null,"evidence_quote":"Establishes rotation-based consistency for active learning in detection, grounding the choice of augmentation."}],"review_version":1}