Pith. sign in

REVIEW 3 major objections 5 minor 59 references

From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Building-MLLM turns indoor point clouds into natural-language recognition, captions, and engineering answers instead of just class labels.

desk verdict Solid domain adaptation of PointLLM for indoor components with a real new dataset and clean ablations; headline scores rest on GPT-family self-evaluation and a 30-sample real transfer, so treat them as synthetic-distribution gains rather than external engineering truth. read the letter →

arxiv 2607.03661 v1 pith:I3R2ULPG submitted 2026-07-04 cs.CV

Shuju Jing , Chao Yin This is my paper · ORCID
classification cs.CV
keywords IndoorBuildingComponentsMultimodalLargeLanguageModelsPointCloudsSyntheticCloud-TextDatasetEngineeringInteractionGeometry-PreservingAlignmentLoRAFine-TuningFacilityOperationandMaintenance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most point-cloud systems for building components stop at a discrete label such as "valve" or "duct bend." That is not enough for facility operation and maintenance: engineers need functional explanations, constraint reasoning, and step-by-step interaction language. This paper claims that a domain-adapted multimodal large language model can close that gap. Building-MLLM takes a component-level point cloud plus a natural-language instruction and answers three escalating tasks: simple recognition, multi-attribute captioning, and multi-engineering question answering that covers knowledge, constraints, spatial relations, and embodied procedures. The authors construct the first large synthetic point-cloud–text instruction set for indoor components (4 198 objects, 47 categories, 37 782 pairs) with a progressive engine that injects ontology priors and engineering constraints. They then add four domain-specific mechanisms—an enhancer that sharpens task-relevant geometry, a regularizer that keeps shallow layers from erasing fine shape detail, a fixed textual prefix that stabilizes domain language, and multi-dimensional LoRA that balances short-range recognition against long-range reasoning. On a held-out synthetic test set the model scores 88 %, 65 % and 68 % (GPT-4 semantic evaluation) on the three tasks, beating the general-domain PointLLM baseline, and it shows initial transfer gains on limited real-world samples.

What carries the argument

The triple-constraint geometry-language alignment (Point Information Enhancer that recursively fuses local and spatial cues, Geometry-Preserving Regularization that reconstructs shallow-layer point features, plus a fixed domain prefix) together with multi-dimensional LoRA that jointly tunes layer range, target modules, rank and progressive scaling; these keep fine geometric distinctions alive while still allowing long-range engineering reasoning.

What would settle it

A larger, independently annotated real-world point-cloud–text benchmark for indoor MEP and architectural components on which Building-MLLM’s GPT-4 scores fall back to or below the general PointLLM baseline, or human expert raters systematically disagree with the GPT-4 scores on engineering logic.

Watch

Extended reading notes

Core claim

A point-cloud-centered multimodal LLM specialized for indoor building components can generate coherent natural-language responses across simple recognition, complex captioning and multi-engineering question answering, provided it is trained on ontology-constrained synthetic instruction data and equipped with geometry-preserving alignment plus multi-dimensional LoRA; under GPT-4 evaluation the resulting Building-MLLM reaches 88.00 %, 65.10 % and 68.14 % on the three tasks and transfers with large gains to small real-world samples.

Load-bearing premise

The progressive synthetic engine that uses ontology templates, multi-view filtering and GPT-4V produces instruction text whose engineering semantics are faithful enough that GPT-4 evaluation of the same model family can serve as a reliable external measure of correctness.

Editorial extensions

If this is right

  • Facility O&M systems can move from discrete class labels to interactive natural-language queries about function, installation constraints and maintenance steps directly from component point clouds.
  • The same progressive engine can be reused to generate instruction data for other concentrated engineering domains that currently lack multimodal corpora.
  • Geometry-preserving regularizers of the kind introduced here become a standard tool when frozen point encoders must be adapted to fine-grained professional categories.
  • Multi-dimensional LoRA schedules that trade short-range geometric grounding against long-range reasoning can be ported to other mixed point–text multimodal models.
  • Component-level language understanding becomes a modular entry point for future scene-level digital-twin agents that reason over connectivity and global constraints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the synthetic-to-real gap can be closed with modest real annotation, the same architecture could serve as a lightweight semantic front-end for existing BIM-CMMS platforms without requiring full scene-level 3D LLMs.
  • The large performance drop on severely incomplete or atypical real scans suggests that active sensing or multi-sensor fusion will be necessary before the method can be trusted in live construction or renovation sites.
  • The seven engineering QA subtypes (especially constraint and embodied interaction) form a natural curriculum that could be used to stress-test other multimodal models claiming domain expertise.
  • Because the authors already freeze most of the 7 B backbone, further distillation to smaller edge-deployable models looks feasible for on-site mobile capture devices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Building-MLLM, a point-cloud-centered multimodal LLM for indoor building components that unifies Simple Recognition, Complex Captioning, and Multi-Engineering QA. It introduces four domain mechanisms (Point Information Enhancer, Geometry-Preserving Regularization on shallow LLM layers, a fixed textual prefix, and multi-dimensional LoRA) to address semantic concentration and multi-task balance, together with a progressive multi-constraint instruction-generation engine that produces a synthetic dataset of 4,198 objects, 47 categories and 37,782 instruction pairs. On a held-out synthetic test set the model reports GPT-4 semantic scores of 88.00 %, 65.10 % and 68.14 % (outperforming a fine-tuned PointLLM baseline), with limited-sample transfer gains on 30 real-world point clouds from Pipework, ScanObjectNN and S3DIS.

Significance. If the reported gains hold under independent evaluation, the work supplies a concrete bridge from discrete geometric labels to language-grounded engineering understanding for facility O&M and digital twins—an underexplored niche relative to general 3D MLLMs. Strengths include a carefully ablated architecture (PIE +5.25 %, GPR +2.12 %, fixed prefix, systematic LoRA layer/module/rank/scaling tables), an open progressive data engine with ontology and cross-modal (BLIP-2/CLIP/Shap-E) filters, public code, and explicit multi-task coverage spanning knowledge, constraint and embodied interaction. These elements make the manuscript a useful methodological foundation even if absolute numbers require stronger external validation.

major comments (3)
  1. §3.1 Step 3 / Table 2 and §5.2: The headline scores (88.00 / 65.10 / 68.14) are GPT-4 judgments of model outputs against reference text generated by GPT-4V under the same ontology and task prompts. Manual review of 800 samples (κ=0.874) only flags binary “need correction” and does not supply independent human-authored ground truth. This creates a distributional loop that can inflate agreement on style and omitted constraints; the paper itself notes residual bias risk. An independent human or code-based engineering-logic audit on a non-trivial subset is required before the superiority claim can be treated as reliable.
  2. §5.3 / Table 12: Real-world transfer is demonstrated on only 30 hand-selected samples (10 per dataset) that remain GPT-4-scored. While the absolute gains over PointLLM are large, the sample size and selection procedure are insufficient to support the claim of “initial generalisability.” Either enlarge the real-world test set with stratified sampling and report confidence intervals, or clearly reframe the transfer results as qualitative feasibility only.
  3. §5.4.1–5.4.2 and Eq. (9)–(10): Ablations convincingly isolate PIE, GPR and LoRA dimensions on the synthetic validation set, yet all Stage-1/Stage-2 metrics ultimately feed into the same GPT-4 evaluator. Without a secondary metric suite (e.g., human preference rankings or external ontology consistency checks) it remains unclear how much of the multi-task balance improvement is genuine geometric-semantic gain versus better imitation of the generator’s distribution.
minor comments (5)
  1. Figure 2 and §3.2: Token-length statistics are given, but category-wise sample counts and class-balance statistics for the 47 categories would help readers assess long-tail effects.
  2. §4.2.2 / Eq. (5)–(7): The recursive-block count and Fourier parameters (α, β) are ablated, yet the precise definition of the gating function G(·) and the residual bottleneck dimensions are only sketched; a short algorithmic box or appendix would improve reproducibility.
  3. Table 9–11: Asterisked rows (independent test set) are helpful, but the distinction between validation-inference and test-inference splits should be stated once in the experimental-setup paragraph for clarity.
  4. Appendix A: Prompt-robustness numbers are reassuring; moving a one-sentence summary into the main evaluation section would strengthen the claim that results are not prompt-specific.
  5. Minor typographical inconsistencies appear (e.g., “generalisability” vs “generalizability”, occasional missing spaces around citations); a final proof-reading pass is recommended.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline GPT-4 scores partly measure agreement with GPT-4V-generated references under shared ontology/task prompts; architecture and gains over PointLLM remain independently trained.

  1. other [Section 3.1 Step 3 + Table 2; Section 5.2 + Tables 6–8]
    "Step 3: Complex Captioning and Multi-Engineering QA Text Instruction-Following Generation. ... we design a Task-Driven Instruction-Following Construction Prompt (Table 2) using API ... GPT-4V ... To address these limitations, we introduce GPT-4 as a high-level semantic evaluator ... we design task-specific standardized GPT-4 evaluation prompts ... Nevertheless, we conservatively acknowledge that some bias may still remain"

    Reference texts for the two complex tasks are generated by GPT-4V under ontology-constrained, task-driven prompts; the same model family (GPT-4) later scores model outputs against those references using prompts that check semantic equivalence to the generated ground truth. Agreement therefore partly reflects shared GPT-family style and prompt distribution rather than fully independent engineering correctness. Manual review and the paper’s caveat reduce but do not break the loop; the Simple Recognition task (template-based) and architectural ablations remain outside it.

full rationale

This is an empirical MLLM paper, not a first-principles derivation. No equations reduce by construction, no fitted parameters are renamed as predictions, and no uniqueness theorems or ansätze are imported via self-citation. The only load-bearing circularity risk is the closed GPT-family loop: Complex Captioning and Multi-Engineering QA references are produced by GPT-4V (Section 3.1 Step 3, Table 2) under the same ontology priors and task-driven prompts later used by the GPT-4 evaluator (Section 5.2, Tables 6–8). Manual review of 800 samples (κ=0.874, ~5 % corrected) and the paper’s own residual-bias caveat mitigate but do not eliminate distributional self-consistency. Real-world transfer (30 samples) and the architectural ablations (PIE/GPR/LoRA) supply independent content, so the central claim is not forced. Score 4 reflects partial circularity confined to the evaluation hinge, not the model itself.

Assumptions & free parameters 4 free parameters · 3 assumptions · 2 invented entities

The central performance claims rest on a handful of hand-chosen architectural hyper-parameters, the assumption that GPT-4V under ontology constraints yields faithful engineering text, and the newly invented modules PIE and GPR whose utility is demonstrated only inside this paper.

free parameters (4)
  • PIE recursive blocks = 3
    Number of recursive Feature/Spatial Adaptation blocks set to 3 after ablation (Table 14); chosen by hand for best Stage-1 accuracy.
  • GPR unfrozen shallow layers = 2
    Number of LLM layers left unfrozen under GPR set to 2 after sensitivity sweep (Table 16).
  • LoRA rank and progressive scaling = r=16, λ=16/32/64
    Rank r=16 and layer-wise λ schedule (16/32/64) selected after multi-dimensional search (Tables 18–20) to balance recognition vs. reasoning.
  • Fourier frequency parameters α,β = α=2.5, β=0.8
    Spatial Adaptation Module frequencies set to α=2.5, β=0.8 after comparison (Table 15).
assumptions (3)
  • domain assumption Ontology-constrained GPT-4V generation under BLIP-2/CLIP/Shap-E view filtering produces instruction text whose engineering semantics are sufficiently accurate for both training and GPT-4 evaluation.
    Stated as the foundation of the progressive engine (Section 3.1) and of the evaluation protocol (Section 5.2).
  • domain assumption Frozen Point-BERT encoder plus a lightweight PIE is sufficient to capture the fine-grained geometric distinctions among 47 indoor component categories.
    Inherited from PointLLM and justified by Stage-1 ablations (Section 4.2).
  • standard math Standard Transformer/LoRA mathematics and MSE reconstruction loss behave as expected under mixed point-text sequences.
    Used without modification for the Projector, GPR loss (Eq. 9) and LoRA update (Eq. 11).
invented entities (2)
  • Point Information Enhancer (PIE)
    purpose: Recursively fuse local geometric and spatial cues into the frozen encoder output before projection.
    Introduced in Section 4.2.2; no independent external validation outside this paper’s ablations.
  • Geometry-Preserving Regularization (GPR)
    purpose: MSE reconstruction constraint on shallow unfrozen LLM layers to prevent geometric erosion.
    Introduced in Section 4.2.3; utility shown only by the paper’s own Stage-1 numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/I3R2ULPG

@misc{pith2026260703661,
  author       = {Pith},
  title        = {Pith review of: From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3R2ULPG}},
  note         = {Machine review of arXiv:2607.03661}
}
read the original abstract

Point cloud-based understanding has become an important enabler for facility operation and maintenance involving indoor building components. However, existing methods output only discrete labels without explaining component functions or natural language interactions. This paper proposes Building-MLLM, a point cloud-centered multimodal large language model (MLLM) for indoor components, which models point clouds and instructions to generate responses across Simple Recognition, Complex Captioning, and Multi-Engineering Question Answering tasks. Building-MLLM addresses semantic concentration through four domain-specific mechanisms: Point Information Enhancer for task-relevant semantics, Geometry-Preserving Regularization preventing geometric erosion, fixed textual prefix for domain stabilization, and multi-dimensional LoRA balancing recognition with reasoning. A multi-constraint progressive instruction-generation engine is developed to compile a synthetic point cloud-text dataset with 4198 objects, 37,782 instruction-following pairs, and 47 categories. Experiments show that Building-MLLM achieves 88.00%, 65.10%, and 68.14% on the three task types, respectively, demonstrating superior indoor component language understanding and providing initial generalizability in transfer inference on other real-world datasets.

Figures

Figures reproduced from arXiv: 2607.03661 by the authors.

Figure 1
Figure 1. Progressive instruction-following generation engine for constructing point cloud-text training data with [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Object category and instruction-following statistics in the proposed dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Examples of point cloud-text instruction-following data for different tasks. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Overview of PointLLM [18] architecture. Building on this foundation, Building-MLLM adopts a two-stage training strategy for special￾ized point cloud–language understanding of indoor building components. In stage 1, we introduce the PIE module, the GPR mechanism, and a …
Figure 5
Figure 5. Figure 5: Building-MLLM architecture for geometry-preserving cross-modal alignment in Stage 1. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Point Information Enhancer (PIE) module for task-oriented geometric enhancement. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Proposed Geometry-Preserving Regularization (GPR) on shallow LLM layers. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Proposed Building-MLLM framework with stage-2 instruction fine-tuning training. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Multi-Dimensional LoRA fine-tuning Strategy for Building-MLLM. [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Test case demonstration of Building-MLLM (1). [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Test case demonstration of Building-MLLM (2). [PITH_FULL_IMAGE:figures/full_fig_p030_11.png]
Figure 12
Figure 12. Figure 12: Inference case studies with GPT-4–based evaluation. [PITH_FULL_IMAGE:figures/full_fig_p031_12.png]
Figure 13
Figure 13. Figure 13: Inference cases transferring Building-MLLM to real-world datasets. [PITH_FULL_IMAGE:figures/full_fig_p033_13.png]
Figure 14
Figure 14. Figure 14: Failure cases of Building-MLLM on real-world point cloud datasets. [PITH_FULL_IMAGE:figures/full_fig_p034_14.png]
Figure 15
Figure 15. Figure 15: Category-level recognition comparison [PITH_FULL_IMAGE:figures/full_fig_p036_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 20 canonical work pages

  1. [1]

    M. Li, V. J. Gan, B. Wang, Integrating hierarchical segmentation and vision-language reason- ing for spatially complex and occluded MEP point clouds, Automation in Construction 179 (2025) 106455.doi:10.1016/j.autcon.2025.106455

    work page doi:10.1016/j.autcon.2025.106455 2025

  2. [3]

    C. Yin, B. Yang, J. C. Cheng, V. J. Gan, B. Wang, J. Yang, Label-efficient semantic segmen- tation of large-scale industrial point clouds using weakly supervised learning, Automation in Construction 148 (2023) 104757.doi:10.1016/j.autcon.2023.104757

    work page doi:10.1016/j.autcon.2023.104757 2023

  3. [4]

    Zhang, L

    Y. Zhang, L. Chen, Q. Lu, Y. Zou, X. Xiahou, S. Sølvsten, C. Hancock, Semi-automated localised updating for as-built BIM of piping systems using point cloud data, Automation in Construction 181 (2026) 106609.doi:10.1016/j.autcon.2025.106609

    work page doi:10.1016/j.autcon.2025.106609 2026

  4. [5]

    A. B. Ersöz, F. Bosché, Evaluating confidence in geometric matching between 3D point clouds and BIM models by integrating coverage, distance, and distribution metrics, Automation in Construction 181 (2026) 106636.doi:10.1016/j.autcon.2025.106636. 41

    doi:10.1016/j.autcon.2025.106636 2026

  5. [6]

    Z. Liu, Y. Mo, B. Hou, M. Li, W. Li, C. Xu, Digital twin modelling approaches and applica- tions in urban infrastructure operations and maintenance, Automation in Construction 179 (2025) 106445.doi:10.1016/j.autcon.2025.106445

    work page doi:10.1016/j.autcon.2025.106445 2025

  6. [7]

    H. Yue, Q. Wang, M. Zhao, Z. Yang, L. Lu, Advancements in digital twin applications for intelligent construction quality management, Journal of Construction Engineering and Man- agement 152 (1) (2026) 03125011.doi:10.1061/JCEMD4.COENG-17126

    work page doi:10.1061/jcemd4.coeng-17126 2026

  7. [8]

    C. Yin, J. C. Cheng, B. Wang, V. J. Gan, Automated classification of piping components from 3D LiDAR point clouds using SE-PseudoGrid, Automation in Construction 139 (2022) 104300.doi:10.1016/j.autcon.2022.104300

    work page doi:10.1016/j.autcon.2022.104300 2022

  8. [9]

    Teclaw, A

    W. Teclaw, A. Tomczak, M. Kasznia, M. Luczkowski, N. Labonnote, E. Hjelseth, ContextNET-Contextual classification of building components using deep neural networks, Advanced Engineering Informatics 67 (2025) 103484.doi:10.1016/j.aei.2025.103484

    work page doi:10.1016/j.aei.2025.103484 2025

Show all 59 references
  1. [10]

    H. Liu, C. Li, Q. Wu, Y. J. Lee, Visual instruction tuning, Advances in neural information processing systems 36 (2023) 34892–34916.doi:10.48550/arXiv.2304.08485

    2023 doi arXiv

  2. [11]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., Gpt-4 technical report, arXiv preprint arXiv:2303.08774 (2023).doi:10.48550/arXiv.2303.08774

    2023 doi arXiv

  3. [12]

    Areerob, V.-Q

    K. Areerob, V.-Q. Nguyen, X. Li, S. Inadomi, T. Shimada, H. Kanasaki, Z. Wang, M. Sug- anuma, K. Nagatani, P.-j. Chun, et al., Multimodal artificial intelligence approaches using large language models for expert-level landslide image analysis, Computer-Aided Civil and Infrastr...

    2025 doi

  4. [13]

    X. Wang, Q. Yue, X. Liu, Crack image classification and information extraction in steel bridges using multimodal large language models, Automation in Construction 171 (2025) 105995.doi:10.1016/j.autcon.2025.105995

    2025 doi

  5. [14]

    P. K.-Y. Wong, J. C. Cheng, C.-F. Chan, P.-H. Leung, X. Tao, et al., Enhancing visual- LLM for construction site safety compliance via prompt engineering and Bi-stage retrieval- augmented generation, Automation in Construction 179 (2025) 106490.doi:10.1016/j. autcon.2025.106490

    2025 doi

  6. [15]

    Armeni, O

    I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, S. Savarese, 3d semantic parsing of large-scale indoor spaces, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1534–1543.doi:10.1109/cvpr.2016.170

    2016 doi

  7. [16]

    C. Yin, B. Wang, V. J. Gan, M. Wang, J. C. Cheng, Automated semantic segmentation of industrial point clouds using ResPointNet++, Automation in Construction 130 (2021) 103874. doi:10.1016/j.autcon.2021.103874

    2021 doi

  8. [17]

    T. Luo, C. Rockwell, H. Lee, J. Johnson, Scalable 3d captioning with pretrained models, Advances in Neural Information Processing Systems 36 (2023) 75307–75337.doi:10.48550/ arXiv.2306.07279

    2023 arXiv

  9. [18]

    R. Xu, X. Wang, T. Wang, Y. Chen, J. Pang, D. Lin, Pointllm: Empowering large language models to understand point clouds, in: European Conference on Computer Vision, Springer, 2024, pp. 131–147.doi:10.1007/978-3-031-72698-9_8. 42

    2024 doi

  10. [19]

    Y. Tang, X. Han, X. Li, Q. Yu, Y. Hao, L. Hu, M. Chen, Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors, in: Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 6617–6626.doi:10.1145/3664647. 3681257

    2024 doi

  11. [20]

    Z. Qi, Y. Fang, Z. Sun, X. Wu, T. Wu, J. Wang, D. Lin, H. Zhao, Gpt4point: A unified framework for point-language understanding and generation, in: Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2024, pp. 26417–26427.doi:10.1109/ cvpr52733.2...

    2024

  12. [21]

    Z. Qi, R. Dong, S. Zhang, H. Geng, C. Han, Z. Ge, L. Yi, K. Ma, Shapellm: Universal 3d object understanding for embodied interaction, in: European Conference on Computer Vision, Springer, 2024, pp. 214–238.doi:10.1007/978-3-031-72775-7_13

    2024 doi

  13. [22]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al., Lora: Low-rank adaptation of large language models., ICLR 1 (2) (2022) 3.doi:10.48550/arXiv. 2106.09685

    2022 doi arXiv

  14. [23]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language su- pervision, in: International conference on machine learning, PmLR, 2021, pp. 8748–8763. doi:10.48...

    2021 doi arXiv

  15. [24]

    J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, in: International conference on machine learning, PMLR, 2023, pp. 19730–19742.doi:10.48550/arXiv.2301.12597

    2023 doi arXiv

  16. [25]

    H. Jun, A. Nichol, Shap-e: Generating conditional 3d implicit functions, arXiv preprint arXiv:2305.02463 (2023).doi:10.48550/arXiv.2305.02463

    2023 doi arXiv

  17. [26]

    Mirzaei, M

    K. Mirzaei, M. Arashpour, E. Asadi, H. Masoumi, Y. Bai, A. Behnood, 3D point cloud data processing with machine learning for construction and infrastructure applications: A comprehensive review, Advanced Engineering Informatics 51 (2022) 101501.doi:10.1016/ j.aei.2021.101501

    2022

  18. [27]

    H. Yue, Q. Wang, X. Nie, H. Fang, J. C. Cheng, S. Jing, B. Wang, Point cloud instance segmentation for building indoor scenes using deep learning and BIM-Generated synthetic point clouds, Advanced Engineering Informatics 72 (2026) 104453.doi:https://doi.org/ 10.1016/j.aei.2026.104453

    2026 doi

  19. [28]

    M. Li, V. J. Gan, K. Li, Vision foundation model with connectivity-based inferencing for zero- shot bridge element segmentation and geometric reconstruction, Automation in Construction 188 (2026) 106995.doi:10.1016/j.autcon.2026.106995

    2026 doi

  20. [29]

    H. Yue, Q. Wang, Y. Huang, H. Fang, J. C. Cheng, L. Li, Enhancing 3D Object Detection for Building Indoor Scenes Using BIM-Generated Synthetic Point Clouds, Journal of Comput- ing in Civil Engineering 40 (4) (2026) 05026003.doi:https://doi.org/10.1061/JCCEE5. CPENG-7008. 43

    2026 doi

  21. [30]

    H. Yue, Q. Wang, L. Cui, C. Li, H. Fang, J. C. Cheng, Transfer Learning for Deep Learning-Based Point Cloud Tasks in Construction Scenes, Journal of Construction Engi- neering and Management 152 (6) (2026) 04026058.doi:https://doi.org/10.1061/JCEMD4. COENG-18262

    2026 doi

  22. [31]

    C. R. Qi, H. Su, K. Mo, L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660.doi:10.1109/cvpr.2017.16

    2017 doi

  23. [32]

    C. R. Qi, L. Yi, H. Su, L. J. Guibas, Pointnet++: Deep hierarchical feature learning on point sets in a metric space, Advances in neural information processing systems 30 (2017). doi:10.48550/arXiv.1706.02413

    2017 doi arXiv

  24. [33]

    H. Zhao, L. Jiang, J. Jia, P. H. Torr, V. Koltun, Point transformer, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16259–16268.doi: 10.1109/iccv48922.2021.01595

    2021 doi

  25. [34]

    B. Koo, R. Jung, Y. Yu, Automatic classification of wall and door BIM element subtypes using 3D geometric deep neural networks, Advanced Engineering Informatics 47 (2021) 101200. doi:10.1016/j.aei.2020.101200

    2021 doi

  26. [35]

    Emunds, N

    C. Emunds, N. Pauen, V. Richter, J. Frisch, C. van Treeck, SpaRSE-BIM: Classification of IFC-based geometry via sparse convolutional neural networks, Advanced Engineering Infor- matics 53 (2022) 101641.doi:10.1016/j.aei.2022.101641

    2022 doi

  27. [36]

    M. A. Uy, Q.-H. Pham, B.-S. Hua, T. Nguyen, S.-K. Yeung, Revisiting point cloud classifi- cation: A new benchmark dataset and classification model on real-world data, in: Proceed- ings of the IEEE/CVF international conference on computer vision, 2019, pp. 1588–1597. doi:10.110...

    2019 doi

  28. [37]

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y. Qiao, W. Ouyang, T. He, H. Zhao, Point transformer v3: Simpler faster stronger, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 4840–4851.doi:10.48550/arXiv.2312. 10035

    2024 doi

  29. [38]

    R. Zhai, J. Zou, Y. He, L. Meng, BIM-driven data augmentation method for semantic seg- mentation in superpoint-based deep learning network, Automation in construction 140 (2022) 104373.doi:10.1016/j.autcon.2022.104373

    2022 doi

  30. [39]

    H. Yue, S. Jing, G. Huang, P. Peng, L. Li, Q. Wang, Weakly supervised framework for semantic segmentation of MEP point clouds, Automation in Construction 187 (2026) 106960. doi:10.1016/j.autcon.2026.106960

    2026 doi

  31. [40]

    Y. Hong, H. Zhen, P. Chen, S. Zheng, Y. Du, Z. Chen, C. Gan, 3D-LLM: Injecting 3D world into large language models, Advances in Neural Information Processing Systems 36 (2023) 20482–20494.doi:10.48550/arXiv.2307.12981

    2023 doi arXiv

  32. [41]

    X. Yu, L. Tang, Y. Rao, T. Huang, J. Zhou, J. Lu, Point-bert: Pre-training 3d point cloud transformers with masked point modeling, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 19313–19322.doi:10.1109/cvpr52688. 2022.01871. 44

    2022 doi

  33. [42]

    W. Dai, J. Li, D. Li, A. Tiong, J. Zhao, W. Wang, B. Li, P. N. Fung, S. Hoi, Instructblip: Towards general-purpose vision-language models with instruction tuning, Advances in neural information processing systems 36 (2023) 49250–49267.doi:10.48550/arXiv.2305.06500

    2023 doi arXiv

  34. [43]

    Deitke, D

    M. Deitke, D. Schwenk, J. Salvador, L. Weihs, O. Michel, E. VanderBilt, L. Schmidt, K. Ehsani, A. Kembhavi, A. Farhadi, Objaverse: A universe of annotated 3d objects, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 13142–13153.d...

    2023 doi

  35. [44]

    Utkucu, H

    D. Utkucu, H. Ying, Z. Wang, R. Sacks, Classification of architectural and MEP BIM objects for building performance evaluation, Advanced Engineering Informatics 61 (2024) 102503. doi:10.1016/j.aei.2024.102503

    2024 doi

  36. [45]

    1179–1189.doi:10.1109/cvpr52729.2023.00120

    L.Xue, M.Gao, C.Xing, R.Martín-Martín, J.Wu, C.Xiong, R.Xu, J.C.Niebles, S.Savarese, Ulip: Learning a unified representation of language, images, and point clouds for 3d un- derstanding, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 202...

    2023 doi

  37. [46]

    S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, E. Chen, A survey on multimodal large language models, National Science Review 11 (12) (2024) nwae403.doi:10.1093/nsr/nwae403

    2024 doi

  38. [47]

    A. B. Sai, A. K. Mohankumar, M. M. Khapra, A survey of evaluation metrics used for nlg systems, ACM Computing Surveys (CSUR) 55 (2) (2022) 1–39.doi:10.1145/3485766

    2022 doi

  39. [48]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318.doi:10.3115/1073083.1073135

    2002 doi

  40. [49]

    Lin, Rouge: A package for automatic evaluation of summaries, in: Text Summarization Branches Out: Proceedings of the ACL Workshop, 2004, pp

    C.-Y. Lin, Rouge: A package for automatic evaluation of summaries, in: Text Summarization Branches Out: Proceedings of the ACL Workshop, 2004, pp. 74–81.doi:10.3115/1220355. 1220369

    2004 doi

  41. [50]

    Banerjee, A

    S. Banerjee, A. Lavie, METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, in: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005, pp. 65– 72.doi:10.3115/12...

    2005 doi

  42. [51]

    Reimers, I

    N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, arXiv preprint arXiv:1908.10084 (2019).doi:10.18653/v1/d19-1410

    1908 doi arXiv

  43. [52]

    T. Gao, X. Yao, D. Chen, Simcse: Simple contrastive learning of sentence embeddings, arXiv preprint arXiv:2104.08821 (2021).doi:10.18653/v1/2021.emnlp-main.552

    2021 doi arXiv

  44. [53]

    Emunds, N

    C. Emunds, N. Pauen, V. Richter, J. Frisch, C. Van Treeck, IFCNet: A benchmark dataset for IFC entity classification, in: Proceedings of the 28th International Workshop on Intelligent Computing in Engineering, EG-ICE, Vol. 166, 2021.doi:10.48550/arXiv.2106.09712

    2021 doi arXiv

  45. [54]

    BIMobject,https://www.bimobject.com/en, accessed: 2025-11-13 (2023)

    2025

  46. [55]

    BIMstore,https://www.bimstore.co/, accessed: 2025-11-13 (2023). 45

    2025

  47. [56]

    NBS Enterprises Ltd., NBS Source,https://source.thenbs.com/, accessed: 2025-11-13 (2023)

    2025

  48. [57]

    Construction Industry Council, CIC BIM Object Library,https://www.bim.cic.hk/en/ resources/bim_objects_list, accessed: 2025-11-13 (2023)

    2025

  49. [58]

    Collins, BIMGEOM (2021).doi:10.7910/DVN/YK86XK

    F. Collins, BIMGEOM (2021).doi:10.7910/DVN/YK86XK

    2021 doi

  50. [59]

    C. Yeo, S. Kim, H. Kim, S. Kim, D. Mun, Deep learning applications in an industrial process plant: repository of segmented point clouds for pipework components, JMST Advances 2 (1) (2020) 15–24.doi:10.1007/s42791-019-00027-y

    2020 doi

  51. [60]

    What is it?

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y. Qiao, P. Gao, H. Li, Pointclip: Point cloud understanding by clip, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 8552–8562.doi:10.48550/arXiv.2112.02413. Appendix A. Promp...

    2022 doi arXiv

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.