{"id":"e35cf764-4a72-4438-8120-b5f135c11164","arxiv_id":"2505.24475","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SPPSFormer uses boundary-accurate, uniform superpoints plus a Transformer-FourierKAN decoder and geometric postprocessing to achieve state-of-the-art roof plane segmentation on RoofN3D and Building3D.","lead":"This paper presents a roof-plane segmentation method that improves a superpoint Transformer by generating better superpoints, adding handcrafted geometric features, and applying geometric postprocessing. It reports state-of-the-art accuracy on two roof point cloud datasets and shows the method tolerates sloppy boundary labels in training, which can reduce labeling cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim is not independently checkable: the Building3D and corrected RoofN3D test labels were created by the authors, so reported gains over DeepRoofPlane/HCBR may reflect label conventions rather than method quality.","rationale":"The paper is a coherent engineering contribution: the ablations in Sec IV-E support the internal claims that the proposed superpoints, handcrafted features, FourierKAN, and postprocessing each help. The internal logic is not the weak point. The headline empirical claim, however, is measured on labels that the authors produced or corrected themselves. Because the method includes geometric plane completion and boundary refinement, the risk is concrete: the 'corrected' labels may encode the same geometric conventions that SPPSFormer's postprocessing applies, which would inflate the comparison against traditional and deep baselines. The reader's weakest_assumption identifies exactly this issue. The concern is testable by independent re-annotation or external evaluation, and no such evidence is currently in the manuscript. I therefore keep the conditional verdict: accept the architecture as plausible, but withhold full verification until the labels and code are public and an independent evaluation is performed.","tokens_in":25690,"tokens_out":5754,"duration_ms":67388,"concrete_test":"Release the test-set annotations and model checkpoints, then have an independent group re-annotate a random subset (e.g., 300 buildings) of the Building3D test set directly from raw point clouds and wireframes using a written protocol with at least two annotators, and independently re-annotate the corrected RoofN3D test samples; report inter-annotator agreement. Rerun DeepRoofPlane and SPPSFormer on these independently produced labels. If the Cov/WCov margins over DeepRoofPlane remain above about 1%, the SOTA claim survives; if the margins fall below or reverse, the author-created labels encode segmentation preferences and the conditional verdict should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion, that SPPSFormer is state-of-the-art on Building3D and on both original and reannotated RoofN3D (Sec IV-D, Tables I-III), depends on the impartiality of the evaluation labels. The Building3D test set was annotated by the authors from wireframe annotations, and the RoofN3D test set was reannotated by the authors after flagging 457 samples as erroneous (Sec IV-A, IV-D). No annotation protocol, inter-annotator agreement, or external validation is reported, and neither the labels nor the code are currently available. Because the same group designed the method, the corrected labels can encode the authors' boundary and plane-completion conventions; if the reannotated RoofN3D labels follow the same geometric completion logic used in SPPSFormer's postprocessing, then HCBR and DeepRoofPlane are scored against a target that is partially aligned with the proposed system's output. The reported margins (Cov +3.03% on original RoofN3D, +1.77% on Building3D over DeepRoofPlane) are plausible but could shrink, reverse, or redistribute under independently produced labels. This is a standard reproducibility/verification gap, not evidence of misconduct; however, it is the load-bearing weak point of both the SOTA claim and the boundary-annotation robustness claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SPPSFormer, a superpoint Transformer for roof plane instance segmentation from point clouds. Its main contributions are: (1) two explicit quality criteria for superpoints and a two-stage generation process combining region growing, local boundary refinement, and K-means; (2) incorporation of handcrafted geometric features (linearity, planarity, scattering, verticality, contour) at the network input; (3) a FourierKAN-based decoder combined with a Transformer; and (4) two algorithm-based postprocessing modules, self-supervised plane completion and boundary refinement. The authors annotated 10,539 buildings from Building3D and corrected RoofN3D annotations. They report state-of-the-art results on the original and reannotated RoofN3D test sets and on Building3D, and they present robustness experiments with boundary-corrupted training annotations and degraded point-cloud quality.","tokens_in":26032,"tokens_out":7865,"duration_ms":89342,"significance":"If the results hold, this is a practically useful contribution for building reconstruction and rooftop photovoltaic planning. The internal evidence is generally strong: the no-improvement ablation row in Table IVa exactly reproduces the SPFormer baseline in Table I, the ablation directions are consistent, and the degradation and boundary-corruption experiments are comprehensive and clearly show advantages over DeepRoofPlane. The two-stage superpoint generation yields large ablation gains (+15.45% Cov on RoofN3D, Table IV), and the hybrid network-plus-postprocessing design gives a plausible mechanism for robustness to boundary annotation noise. The authors also promise to release code, trained models, and datasets, which would make the empirical claims checkable. The main limitation is that the evaluation labels for Building3D and the reannotated RoofN3D test set were created by the authors without an independent verification protocol, and all experiments use a single seed without variance estimates.","major_comments":[{"comment":"The overall SOTA claim is not yet independently checkable. The Building3D test set was annotated by the authors from wireframe annotations, and the RoofN3D test set was reannotated by the authors after flagging 457 erroneous samples. No annotation protocol, inter-annotator agreement, or external validation is reported, and the labels, code, and trained models are promised but not yet available. The original RoofN3D comparison in Table I is external and therefore less affected, but the reannotated RoofN3D (Table III) and Building3D (Table II) comparisons can encode the authors' geometric conventions, especially because the method's Plane Completion and Boundary Refinement modules (Sec. III-D) implement similar geometric logic. Please supply the release artifacts, a detailed annotation protocol, and an independent re-annotation agreement study, or evaluate on an externally labeled subset.","section":"Sec. IV-A and IV-D, Tables I-III"},{"comment":"All experiments use a single fixed random seed (200), and no error bars or significance tests are reported. The Cov margins over DeepRoofPlane are 3.03% (original RoofN3D), 1.77% (Building3D), and 2.57% (reannotated RoofN3D). These margins are plausible but cannot be distinguished from seed-to-seed variation without multiple runs. Please report means and standard deviations over at least 3-5 seeds, or equivalent bootstrap intervals, for at least the main tables.","section":"Sec. IV-C and Tables I-III"},{"comment":"The plane completion module estimates its region-growing thresholds per test cloud from the network's own predicted plane instances, and it contributes a large +7.29% Cov gain on RoofN3D (Table V). Because HCBR and DeepRoofPlane do not have an analogous per-cloud adaptive postprocessing step, the comparison may not be strictly apples-to-apples. Please provide a sensitivity analysis of the threshold estimation (e.g., fixed global thresholds versus inferred thresholds, and sensitivity to the number of predicted planes) and clarify why the inferred thresholds cannot overfit to network errors.","section":"Sec. III-D1 and Table V"},{"comment":"The boundary-robustness claim is supported by a single corruption level (swapping labels of points closer than 0.5 m). Since this robustness is a headline contribution, please report results at multiple corruption levels and with different corruption mechanisms (e.g., dilating or eroding boundaries), and state the fraction of boundary points actually swapped. Without this, the claim that the model is 'not sensitive to boundary annotations' is under-supported.","section":"Sec. IV-F and Tables X-XI"}],"minor_comments":[{"comment":"The caption mentions 'Red dotted lines highlight representative problem areas,' but no such marks are visible or explained; please add visible annotations or a legend.","section":"Fig. 6"},{"comment":"The relationship among the 457 erroneous samples (Sec. IV-A), the 49 reannotated test samples (Sec. IV-D), and the 408 reannotated training/validation samples should be stated explicitly (49 + 408 = 457) to avoid misreading.","section":"Sec. IV-A and IV-D"},{"comment":"Equation (1) defines n as the desired average number of points per superpoint, but Eq. (2) uses n as a threshold; please clarify the relationship and report the actual value of n used in the experiments, since it is a free parameter.","section":"Eqs. (1)-(2)"},{"comment":"The contour feature's angle threshold is user-defined but no value is given; please report it and any sensitivity analysis.","section":"Sec. III-B"},{"comment":"Tables IV and V would be more self-contained if they explicitly stated that all rows exclude postprocessing unless marked; currently the reader must infer this from the text.","section":"Tables IV-V"},{"comment":"The QTPS evaluations in Tables I and II cover only 392 and 901 of the 1,000 test samples; the table captions should state this, as the text does.","section":"Tables I-II"},{"comment":"The implementation details should report voxel size, superpoint size n, number of decoder layers and heads, and the exact handcrafted feature dimensions for reproducibility.","section":"Sec. IV-C"}],"recommendation":"major_revision","confidential_remarks":"No additional concerns beyond the report. The main risk is label independence; I would ask for a concrete release plan or independent evaluation before final acceptance. The single-seed issue compounds this. These are fixable within the manuscript's scope, hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the two-stage superpoint generation (region growing plus local boundary relabeling, then K-means size equalization) is a genuinely useful idea, and the ablations show it drives most of the gain. Second, the SOTA claim is only partly checkable today: the original RoofN3D part stands on an external benchmark, but the Building3D and reannotated-RoofN3D results rest on labels the authors made themselves, with no annotation protocol or inter-rater numbers.\n\nWhat's new and good: the paper establishes two sensible criteria for superpoints in Transformers—accurate boundaries and uniform size/shape—and shows that violations hurt. The K-means equalization step is simple and effective. The boundary-corruption experiment is the most interesting result: retraining with intentionally degraded boundary labels barely hurts SPPSFormer while DeepRoofPlane drops a lot. That is a practical finding for anyone annotating roof datasets. The ablation tables are internally consistent; the no-improvement row reproduces the SPFormer baseline, so the pipeline is believable. They also build a 10,539-building roof instance dataset from Building3D wireframes, which will be useful if released.\n\nSoft spots: all numbers are single-seed, no error bars, so the 1-3% Cov margins over DeepRoofPlane are not statistically grounded. The author-created test labels for Building3D and corrected RoofN3D are the load-bearing weakness: if those labels encode the authors' geometric completion conventions, part of the reported gain may be label alignment. The plane completion module infers region-growing thresholds from the network's own predictions on each test cloud; that is disclosed, but it does open the door to using test-set statistics. The boundary degradation is simulated with a single 0.5 m label swap, and the robustness claim is only tested against one deep baseline. None of this suggests anything dishonest; it just means the SOTA claim needs independent labels and code/data before it is verified.\n\nBottom line: this is a solid, engineering-heavy submission. It deserves peer review, but the reviewers should push for code/data release and an external or independent evaluation of the new labels. The superpoint criteria and the boundary-robustness result are worth citing regardless.","headline":"Useful superpoint-generation trick and an interesting boundary-robustness result, but the SOTA claim depends on author-produced labels that need independent verification.","tokens_in":26562,"tokens_out":2159,"would_cite":true,"duration_ms":23744,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SPPSFormer claims state-of-the-art roof plane instance segmentation on RoofN3D and Building3D by combining two-stage high-quality superpoint generation with a superpoint Transformer, handcrafted features, a FourierKAN-augmented decoder…","keywords":["roof plane instance segmentation","superpoint Transformer","3D point cloud segmentation","superpoint generation","FourierKAN","plane boundary refinement","point cloud quality degradation","RoofN3D dataset"],"falsifier":"If an independent group reannotates RoofN3D and Building3D test sets following only geometric plane definitions, with annotators blind to algorithm outputs, and the reported Cov/WCov advantage of SPPSFormer over DeepRoofPlane shrinks or reverses on those independently labeled sets, the claim of general superiority would be falsified. A second falsifier: corrupting training-set boundary labels with a different protocol, such as symmetric label noise across all instances, and showing that SPPSFormer's Cov drop exceeds the reported small decline would falsify the boundary-insensitivity claim.","tokens_in":25512,"feed_emoji":"🏠","tokens_out":5573,"duration_ms":65554,"temperature":0.7,"pith_summary":"The paper proposes SPPSFormer, a superpoint Transformer for segmenting roof plane instances from 3D point clouds. The central claim is that generating superpoints with two properties, accurate boundaries and uniform size and shape, together with handcrafted geometric features and a FourierKAN-based decoder, yields state-of-the-art accuracy on RoofN3D and Building3D. A second claim is that the hybrid design, deep network followed by traditional postprocessing, makes the model largely insensitive to corrupted plane boundary annotations during training, substantially reducing annotation burden. The authors also reannotate parts of RoofN3D, annotate a new Building3D roof-plane test set, and report that point cloud density, density uniformity, and 3D point precision strongly affect segmentation performance. If correct, the work lowers the cost of building training data and highlights data-quality-aware augmentation as a key robustness direction.","feed_headline":"Two-stage superpoints push roof-plane segmentation to new SOTA","feed_subtitle":"Hybrid deep-plus-traditional pipeline stays accurate even when training boundary labels are corrupted.","key_machinery":"The central object is the high-quality superpoint, a spatially coherent point group that does not mix instances at boundaries and is roughly uniform in size and shape, so the Transformer sees consistent input tokens. The load-bearing mechanism is the two-stage generation pipeline: coarse region growing with strict parameters plus local boundary relabeling, then K-means splitting of each coarse superpoint and of the unfitted noise set into a target number of clusters determined by a desired average points-per-superpoint. The decoder replaces the mask-branch MLP with FourierKAN, and traditional postprocessing, self-supervised plane completion that infers region-growing thresholds from already segmented planes plus a composite-distance boundary refinement, corrects the network outputs.","core_discovery":"SPPSFormer is a superpoint Transformer whose two-stage superpoint generation first produces boundary-accurate superpoints via region growing plus local boundary optimization, then splits them with K-means to enforce uniform size and shape. Feeding these superpoints into a 3D U-Net with handcrafted features and a FourierKAN-Transformer decoder, followed by self-supervised plane completion and efficient boundary refinement, achieves state-of-the-art coverage, weighted coverage, F1, and accuracy on the original and reannotated RoofN3D test sets and on the authors' Building3D roof-plane dataset. Critically, the paper claims the model is largely insensitive to plane-boundary annotation errors in the training set: retraining with boundary labels randomly swapped between nearby planes drops DeepRoofPlane's performance sharply while SPPSFormer degrades only slightly. The paper also reports that reducing point density, increasing density variation, or degrading 3D point precision on Building3D substantially lowers accuracy for both methods, with SPPSFormer consistently ahead.","pith_inferences":["Editorial: The reported gains over DeepRoofPlane may partly reflect label alignment, because the authors reannotated RoofN3D samples they judged erroneous and annotated the Building3D test set themselves; an independent third-party labeling study would separate method quality from label preference.","Editorial: The self-supervised plane completion module assumes the network already segmented most planes correctly; its margin over no postprocessing should be measured on low-quality subsets rather than averaged over the full test set to test this assumption.","Editorial: The FourierKAN benefit could be sharpened by comparing against an MLP of matched parameter count and depth under the same superpoint generation, since the ablation changes architecture while keeping the rest of the pipeline fixed.","Editorial: A testable extension is applying the same two-stage superpoint generation to generic indoor instance segmentation benchmarks, which would show whether the size and shape uniformity criterion transfers beyond roofs."],"forward_implications":["Roof-plane segmentation accuracy on RoofN3D and Building3D should be reproducible at the reported SOTA level from the released code, trained models, and datasets.","Training sets for plane instance segmentation need only accurate plane main bodies, not fine boundary labels, which cuts annotation cost substantially.","Point cloud quality factors, density level, density uniformity, and point precision, should be treated as first-order experimental variables, and augmentations simulating these degradations should improve robustness on low-quality lidar.","The two superpoint criteria and the two-stage generation recipe can be transferred to other superpoint Transformer instance segmentation tasks."],"supporting_citations":[{"why":"Provides the base superpoint Transformer architecture (SPFormer) that SPPSFormer modifies and the strongest transformer baseline compared in the experiments.","marker":"[14]"},{"why":"Supplies the local boundary relabeling method used in the first stage of superpoint generation and the HCBR traditional baseline.","marker":"[2]"},{"why":"DeepRoofPlane is the current deep-learning SOTA competitor that the paper compares against and retrains in boundary-corruption and point-quality degradation experiments.","marker":"[22]"},{"why":"RoofN3D is the dataset the paper evaluates on, partially reannotates, and corrects to enable the experimental claims.","marker":"[25]"},{"why":"Building3D supplies the wireframe annotations from which the authors create their new 10,539-building roof plane instance segmentation dataset.","marker":"[26]"},{"why":"Defines the KAN architecture that the decoder adapts for mask feature extraction.","marker":"[24]"},{"why":"Provides the FourierKAN variant used in the decoder to replace the MLP, claimed to improve representational capacity at matched parameter count.","marker":"[65]"},{"why":"The graph-based energy optimization superpoint generation method used by SPFormer; the ablation compares these 'basic superpoints' against the proposed two-stage output.","marker":"[57]"}],"fun_headline_variants":["Two-stage superpoints lift roof plane segmentation to SOTA","Boundary-accurate superpoints improve roof instance segmentation","SPPSFormer: robust roof plane segmentation via superpoint transformer","New superpoint method tops roof plane segmentation benchmarks","Roof plane segmentation SOTA with uniform superpoints"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art result depends on the correctness and impartiality of the evaluation labels, because the authors reannotated RoofN3D test samples they judged erroneous and annotated the Building3D test set themselves from wireframes.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage superpoints lift roof plane segmentation to SOTA","Boundary-accurate superpoints improve roof instance segmentation","SPPSFormer: robust roof plane segmentation via superpoint transformer","New superpoint method tops roof plane segmentation benchmarks","Roof plane segmentation SOTA with uniform superpoints"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000502,"raw_usage":{"total_tokens":2500,"prompt_tokens":1041,"completion_tokens":1459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":1380}},"tokens_in":657,"tokens_out":1459,"duration_ms":14207,"temperature":1.0,"reasoning_tokens":1380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:21:47.820268+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If an independent group reannotates RoofN3D and Building3D test sets following only geometric plane definitions, with annotators blind to algorithm outputs, and the reported Cov/WCov advantage of SPPSFormer over DeepRoofPlane shrinks or reverses on those independently labeled sets, the claim of general superiority would be falsified. A second falsifier: corrupting training-set boundary labels with a different protocol, such as symmetric label noise across all instances, and showing that SPPSFormer's Cov drop exceeds the reported small decline would falsify the boundary-insensitivity claim.","supporting_citations":[{"cited_title":"Superpoint transformer for 3d scene instance segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the base superpoint Transformer architecture (SPFormer) that SPPSFormer modifies and the strongest transformer baseline compared in the experiments."},{"cited_title":"A boundary-aware point clustering approach in euclidean and embedding spaces for roof plane segmentation,","cited_arxiv_id":null,"evidence_quote":"DeepRoofPlane is the current deep-learning SOTA competitor that the paper compares against and retrains in boundary-corruption and point-quality degradation experiments."},{"cited_title":"Roofn3d: A database for 3d building reconstruction with deep learning,","cited_arxiv_id":null,"evidence_quote":"RoofN3D is the dataset the paper evaluates on, partially reannotates, and corrects to enable the experimental claims."},{"cited_title":"Building3d: A urban-scale dataset and benchmarks for learning roof structures from point clouds,","cited_arxiv_id":null,"evidence_quote":"Building3D supplies the wireframe annotations from which the authors create their new 10,539-building roof plane instance segmentation dataset."},{"cited_title":"Fourierkan-gcf: Fourier kolmogorov-arnold network – an effective and efficient feature transformation for graph collaborative filtering,","cited_arxiv_id":null,"evidence_quote":"Provides the FourierKAN variant used in the decoder to replace the MLP, claimed to improve representational capacity at matched parameter count."},{"cited_title":"Large-scale point cloud semantic segmentation with superpoint graphs,","cited_arxiv_id":null,"evidence_quote":"The graph-based energy optimization superpoint generation method used by SPFormer; the ablation compares these 'basic superpoints' against the proposed two-stage output."}],"review_version":1}